Hybrid 2D-to-3D localization in changing environments

Co-PI: Burcu Akinci, Professor, Civil and Environmental Engineering, College of Engineering; Associate Dean for Research, College of Engineering

In current visual SLAM applications, localization performance is highly dependent on the ever-changing nature of environmental conditions (illumination, seasons, etc.) and the shape of structures can affect accuracy. Accurate localization can only be achieved in static environments and cannot be applied in large-scale natural environments. Contractors document their progress with pictures, however, the location of these pictures has to be specified manually in a 3D digital twin model; which is rarely performed due to the labor-intensive nature of it. To address the above challenges, we aim to develop a hybrid 2D-3D localization method based on our previous work that explored geometric and data-driven approaches [1~8]:

• First, we will gather an initial 3D map and CAD model as well as video data with ground truth from two construction sites at different stages of completion with the help of our industry partners.
• Second, with the gathered datasets, we will compare the localization robustness and accuracy of our previously developed data-driven place recognition method and the 2D-3D line-based localization method.
• Thirdly, we will combine the advantages of both data-driven and classical geometry based approaches, and design and evaluate a hybrid 2D-3D localization method that is robust to changing environmental conditions, viewpoints and geometry structures and combines advantages of both types of approaches.

In our video, we show short clips of our previously developed geometric and data-driven method.