OSMLoc: Single Image-Based Visual Localization in OpenStreetMap with Fused Geometric and Semantic Guidance

Youqi Liao*¹ Xieyuanli Chen*² shuhao Kang³ Jianping Li^4,†
Zhen Dong¹ Hongchao Fan⁵ Bisheng Yang¹
¹Wuhan University ² National University of Defense Technology ³Technical University of Munich
⁴ Nanyang Technological University ⁵ Norwegian University of Science and Technology ^*The first two authors contribute equally. ^†Corresponding author.

[Paper] [Video] [Code] [BibTeX]

What can OSMLoc do?

OSMLoc is an image-to-OpenstreetMap (I2O) visual localization framework with geometric and semantic guidance. (a) shows the core idea of our method that integrates the geometry and semantic guidance into the framework, while (b) shows the worldwide evaluation results.

Abstract

OpenStreetMap (OSM), a rich and versatile source of volunteered geographic information (VGI), facilitates human self-localization and scene understanding by integrating nearby visual observations with vectorized map data. However, the disparity in modalities and perspectives poses a major challenge for effectively matching camera imagery with compact map representations, thereby limiting the full potential of VGI data in real-world localization applications. Inspired by the fact that the human brain relies on the fusion of geometric and semantic understanding for spatial localization tasks, we propose the OSMLoc in this paper. OSMLoc is a brain-inspired visual localization approach based on first-person-view images against the OSM maps. It integrates semantic and geometric guidance to significantly improve accuracy, robustness, and generalization capability. First, we equip the OSMLoc with the visual foundational model to extract powerful image features. Second, a geometry-guided depth distribution adapter is proposed to bridge the monocular depth estimation and camera-to-BEV transform. Thirdly, the semantic embeddings from the OSM data are utilized as auxiliary guidance for image-to-OSM feature matching. To validate the proposed OSMLoc, we collect a worldwide cross-area and cross-condition (CC) benchmark for extensive evaluation. Experiments on the MGL dataset, CC validation benchmark, and KITTI dataset have demonstrated the superiority of our method.

Introduction Video

Localization results

Comparable results

Qualitative results

Application in Sequential loaclization

BibTex

 @article{liao2024osmloc,

    title={OSMLoc: Single Image-Based Visual Localization in OpenStreetMap
    with Fused Geometric and Semantic Guidance},

    author={Liao, Youqi and Chen, Xieyuanli and Kang, Shuhao and Li, Jianping and 

    Dong, Zhen and Fan, Hongchao and Yang, Bisheng},

    journal={arXiv preprint arXiv:2411.08665},

    year={2024}

  }

Acknowledgements: We borrow this template from FreeReg.