OSMLoc: Single Image-Based Visual Localization in OpenStreetMap with Semantic and Geomet- ric Guidances

Youqi Liao*¹ Xieyuanli Chen*² shuhao Kang³ Jianping Li^4,†
Zhen Dong¹ Hongchao Fan⁵ Bisheng Yang^1
1Wuhan University ² National University of Defense Technology ³Technical University of Munich
⁴ Nanyang Technological University ⁵ Norwegian University of Science and Technology ^*The first two authors contribute equally.    ^†Corresponding author.   

[Paper]      [Video]     [Code]     [BibTeX]

What can OSMLoc do?

OSMLoc is an image-to-OpenstreetMap (I2O) visual localization framework with geometric and semantic guidance. (a) shows the core idea of our method that integrates the geometry and semantic guidance into the framework, while (b) shows the worldwide evaluation results.

Abstract

OpenStreetMap (OSM), an online and versatile source of volunteered geographic information (VGI), is widely used for human self-localization by matching nearby visual observations with vectorized map data. However, due to the divergence in modalities and views, image-to-OSM (I2O) matching and localization remain challenging for robots, preventing the full utilization of VGI data in the unmanned ground vehicles and logistic industry. Inspired by the fact that the human brain relies on different regions when processing geometric and semantic information for spatial localization tasks, in this paper, we propose the OSMLoc. OSMLoc is a brain-inspired monocular visual localization method with semantic and geometric guidance to improve accuracy, robustness, and generalization ability. First, we equip the OSMLoc with the visual foundational model to extract powerful image features. Second, a geometry-guided depth distribution adapter is proposed to bridge the monocular depth estimation and camera-to-BEV transform. Thirdly, the semantic embeddings from the OSM data are utilized as auxiliary guidance for image-to-OSM feature matching. To validate the proposed OSMLoc, we collect a worldwide cross-area and cross-condition (CC) benchmark for extensive evaluation. Experiments on the MGL dataset, CC validation benchmark, and KITTI dataset have demonstrated the superiority of our method.

Introduction Video

We are working hard to make the introduction video.

Localization results

Comparable results

Qualitative results

Application in Sequential loaclization

BibTex

@article{liao2024osmloc,
  title={OSMLoc: Single Image-Based Visual Localization in OpenStreetMap with Geometric and Semantic Guidances},
  author={Liao, Youqi and Chen, Xieyuanli and Kang, Shuhao and Li, Jianping and
  Dong, Zhen and Fan, Hongchao and Yang, Bisheng},
  journal={arXiv preprint arXiv:2411.08665},
  year={2024}
}

Acknowledgements: We borrow this template from FreeReg.