UniPR-3D

Model Card for UniPR-3D

UniPR-3D is a universal visual place recognition (VPR) framework that supports both single-frame and sequence-to-sequence matching. It leverages 3D visual geometry grounded tokens within a transformer architecture to produce robust, viewpoint-invariant descriptors for long-term place recognition under challenging environmental variations (e.g., seasonal, weather, lighting, and viewpoint changes).

Model Details

Model Description

• Developed by: Tianchen Deng, Xun Chen, Ziming Li, Hongming Shen, Danwei Wang, Javier Civera, Hesheng Wang
• Shared by: Tianchen Deng
• Model type: Vision Transformer with 3D-aware token aggregation for visual place recognition
• Language(s): English (dataset metadata); model is vision-only
• License: MIT

Model Sources

• Repository: repo
• Paper: UniPR-3D: Towards Universal Visual Place Recognition with 3D Visual Geometry Grounded Transformer (arXiv:2512.21078, 2025)
• Demo: No demo available

Uses

Direct Use

This model can be used out-of-the-box to extract compact, discriminative global descriptors from:

• Single RGB images (for frame-to-frame VPR)
• Sequences of images (for sequence-to-sequence VPR)

These descriptors are suitable for large-scale localization, robot navigation, and SLAM systems requiring robustness to appearance changes.

Downstream Use

• Integration into visual SLAM or long-term autonomous navigation pipelines
• Replacement for traditional VPR backbones (e.g., NetVLAD, MixVPR, EigenPlaces)
• Fine-tuning on domain-specific datasets (e.g., underground, aerial, or underwater environments)

Out-of-Scope Use

• Not intended for real-time inference on low-power embedded devices without optimization (latency ~8.23 ms on RTX 4090)
• Not designed for non-visual modalities (e.g., LiDAR, audio, text)
• Performance may degrade in extreme occlusion, textureless scenes, or indoor environments not seen during training

Bias, Risks, and Limitations

• Trained primarily on urban street-level imagery (GSV-Cities, Mapillary MSLS), so generalization to rural, indoor, or non-Western cities may be limited
• Inherits biases from training data (e.g., geographic overrepresentation of North America/Europe)
• No explicit fairness or demographic considerations (as it is a geometric vision model)

Recommendations

• Evaluate on target domain before deployment
• Monitor recall performance on your specific dataset using standard VPR metrics (R@1, R@5)

How to Get Started with the Model

The exact inference script is provided in the GitHub repo (eval_lora.py, main_ft.py). Pretrained weights are available on Hugging Face or via the repo release.

Training Details

Training Data

• Single-frame model: Trained on GSV-Cities
• Multi-frame model: Trained on Mapillary Street-Level Sequences (MSLS)
• Both datasets contain millions of geo-tagged urban street-view images across diverse cities, seasons, and conditions.

Training Procedure

Preprocessing

• Images resized to 518×518
• Sequences sampled with spatial proximity for multi-frame training

Training Hyperparameters

• Backbone: DINOv2 (ViT-large)
• Optimization: AdamW, learning rate scheduling
• Loss: Multi-similarity loss with pair weighting
• Training regime: Mixed-precision (fp16) on NVIDIA GPUs

Speeds, Sizes, Times

• Inference latency: Single frame - 8.23 ms per image (RTX 4090)
• Descriptor dimension: 17152 (for UniPR-3D)
• Training time: Not disclosed (multi-day runs on multi-GPU setup)

Evaluation

Testing Data, Factors & Metrics

Testing Data

• Single frame evaluation:
  • <a href="https://codalab.lisn.upsaclay.fr/competitions/865">MSLS Challenge</a>, where you upload your predictions to their server for evaluation.
  • Single-frame <a href="https://www.mapillary.com/dataset/places">MSLS</a> Validation set
  • Nordland dataset, <a href="https://data.ciirc.cvut.cz/public/projects/2015netVLAD/Pittsburgh250k/">Pittsburgh</a> dataset and SPED dataset, you may download them from <a href="https://surfdrive.surf.nl/index.php/s/sbZRXzYe3l0v67W">here</a>, aligned with DINOv2 SALAD.
• Multi-frame evaluation:
  • Multi-frame <a href="https://www.mapillary.com/dataset/places">MSLS</a> Validation set
  • Two sequence from <a href="https://robotcar-dataset.robots.ox.ac.uk/datasets/">Oxford RobotCar</a>, you may download them <a href="https://entuedu-my.sharepoint.com/personal/heshan001_e_ntu_edu_sg/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Fheshan001%5Fe%5Fntu%5Fedu%5Fsg%2FDocuments%2Fcasevpr%5Fdatasets%2Foxford%5Frobotcar&viewid=e5dcb0e9%2Db23f%2D44cf%2Da843%2D7837d3064c2e&ga=1">here</a>.
    • 2014-12-16-18-44-24 (winter night) query to 2014-11-18-13-20-12 (fall day) db
    • 2014-11-14-16-34-33 (fall night) query to 2015-11-13-10-28-08 (fall day) db
  • <a href="https://github.com/gmberton/VPR-datasets-downloader/blob/main/download_nordland.py">Nordland (filtered) dataset</a>

Factors

• Seasonal variation (summer ↔ winter)
• Day vs. night
• Weather (sunny, rainy, snowy)
• Viewpoint change (lateral shift, orientation)

Metrics

• Recall@K (R@1, R@5, R@10): Standard metric for VPR – fraction of queries with correct match in top-K retrieved database images

Results

Summary

Our method achieves significantly higher recall than competing approaches, achieving new state-of-the-art performance on both single and multiple frame benchmarks.

Single-frame matching results

<style> table, th, td { border-collapse: collapse; text-align: center; } </style> <table> <tr> <th colspan="2"></th> <th colspan="2">MSLS Challenge</th> <th colspan="2">MSLS Val</th> <th colspan="2">NordLand</th> <th colspan="2">Pitts250k-test</th> <th colspan="2">SPED</th> </tr> <tr> <th>Method</th> <th>Latency (ms)</th> <th>R@1</th> <th>R@5</th> <th>R@1</th> <th>R@5</th> <th>R@1</th> <th>R@5</th> <th>R@1</th> <th>R@5</th> <th>R@1</th> <th>R@5</th> </tr> <tr> <td>MixVPR</td> <td>1.37</td> <td>64.0</td> <td>75.9</td> <td>88.0</td> <td>92.7</td> <td>58.4</td> <td>74.6</td> <td>94.6</td> <td><u>98.3</u></td> <td>85.2</td> <td>92.1</td> </tr> <tr> <td>EigenPlaces</td> <td>2.65</td> <td>67.4</td> <td>77.1</td> <td>89.3</td> <td>93.7</td> <td>54.4</td> <td>68.8</td> <td>94.1</td> <td>98.0</td> <td>69.9</td> <td>82.9</td> </tr> <tr> <td>DINOv2 SALAD</td> <td>2.41</td> <td><u>73.0</u></td> <td><u>86.8</u></td> <td><u>91.2</u></td> <td><u>95.3</u></td> <td><u>69.6</u></td> <td><u>84.4</u></td> <td><u>94.5</u></td> <td><b>98.7</b></td> <td><u>89.5</u></td> <td><u>94.4</u></td> </tr> <tr> <td>UniPR-3D (ours)</td> <td>8.23</td> <td><b>74.3</b></td> <td><b>87.5</b></td> <td><b>91.4</b></td> <td><b>96.0</b></td> <td><b>76.2</b></td> <td><b>87.3</b></td> <td><b>94.9</b></td> <td>98.1</td> <td><b>89.6</b></td> <td><b>94.5</b></td> </tr> </table>

Sequence matching results

<table> <tr> <th></th> <th colspan="3">MSLS Val</th> <th colspan="3">NordLand</th> <th colspan="3">Oxford1</th> <th colspan="3">Oxford2</th> </tr> <tr> <th>Method</th> <th>R@1</th> <th>R@5</th> <th>R@10</th> <th>R@1</th> <th>R@5</th> <th>R@10</th> <th>R@1</th> <th>R@5</th> <th>R@10</th> <th>R@1</th> <th>R@5</th> <th>R@10</th> </tr> <tr> <td>SeqMatchNet</td> <td>65.5</td> <td>77.5</td> <td>80.3</td> <td>56.1</td> <td>71.4</td> <td>76.9</td> <td>36.8</td> <td>43.3</td> <td>48.3</td> <td>27.9</td> <td>38.5</td> <td>45.3</td> </tr> <tr> <td>SeqVLAD</td> <td>89.9</td> <td>92.4</td> <td>94.1</td> <td>65.5</td> <td>75.2</td> <td>80.0</td> <td>58.4</td> <td>72.8</td> <td>80.8</td> <td>19.1</td> <td>29.9</td> <td>37.3</td> </tr> <tr> <td>CaseVPR</td> <td><u>91.2</u></td> <td><u>94.1</u></td> <td><u>95.0</u></td> <td><u>84.1</u></td> <td><u>89.9</u></td> <td><u>92.2</u></td> <td><u>90.5</u></td> <td><u>95.2</u></td> <td><u>96.5</u></td> <td><u>72.8</u></td> <td><u>85.8</u></td> <td><u>89.9</u></td> </tr> <tr> <td>UniPR-3D (ours)</td> <td><b>93.7</b></td> <td><b>95.7</b></td> <td><b>96.9</b></td> <td><b>86.8</b></td> <td><b>91.7</b></td> <td><b>93.8</b></td> <td><b>95.4</b></td> <td><b>98.1</b></td> <td><b>98.7</b></td> <td><b>80.6</b></td> <td><b>90.3</b></td> <td><b>93.9</b></td> </tr> </table>

Compute Infrastructure

Hardware

• NVIDIA RTX 4090

Software

• Python 3.11.10 + CUDA 12.1
• Based on SALAD and VGGT

Citation

BibTeX:

@article{deng2025unipr3d,
  title={UniPR-3D: Towards Universal Visual Place Recognition with 3D Visual Geometry Grounded Transformer},
  author={Deng, Tianchen and Chen, Xun and Li, Ziming and Shen, Hongming and Wang, Danwei and Civera, Javier and Wang, Hesheng},
  journal={arXiv preprint arXiv:2512.21078},
  year={2025}
}

APA: Deng, T., Chen, X., Li, Z., Shen, H., Wang, D., Civera, J., & Wang, H. (2025). UniPR-3D: Towards Universal Visual Place Recognition with 3D Visual Geometry Grounded Transformer. arXiv preprint arXiv:2512.21078.

Contact

For questions, pretrained model access, or qualitative comparisons, please contact:
📧 Tianchen Deng – [email protected]

---

📌 Acknowledgement: This implementation builds upon SALAD and VGGT. Please cite those works if you use their components.