Affordable Access

Towards geometric and semantic change detection for mixed reality experiences

  • Roupin, Olivier
Publication Date
Sep 22, 2023
External links


Photorealistic Augmented Reality (AR) –or “Mixed Reality”– is applicable to various immersive experiences, from entertainment to simulation-based training, and for previsualization tasks. The realistic integration of virtual elements requires an accurate and up-to-date model of the real scene's geometry and semantic contents. The 3D structure of the environment impacts the visual rendering as well as the spatial layout of the virtual content and its interactions with real objects. Given the hardware constraints of widely available AR devices, it is impractical to thoroughly scan the geometry of the scene with each use. Still, some the scene's structural information will remain constant over time only requiring local updates to accurately represent the scene's current state. The aim of this doctoral study has been to provide the means to identify and correct areas of change in a scene through detection of inconsistencies between a prior representation and an current observations. In this thesis, we present the completion of a lightweight reprojection-based geometric change detection framework. We also present the elaboration of a graph-based semantic scene model for Mixed Reality, and a method for its generation from semantic analysis. This model will then be used in a 3D scene registration system, in order to test its applicability to the localization task in AR.

Report this publication


Seen <100 times