CityPulse-DINOv2

CityPulse

[Paper] [Citing] [Project Page] [Github Repository]

Model Summary

CityPulse-DINOv2 is an urban structural change detection model finetuned based on the parameters of a Siamese DINOv2 (ViT-B/14) model. This model has been fine-tuned on the CityPulse (Huang et al. 2024) dataset. It achieves an accuracy of 88.85% in the urban building change detection task, which makes it suitable for fine-grained assessment of urban changes. The intention behind releasing this model is to provide the research community with more comprehensive and extensive resources, facilitate broader downstream research on urbans, and advance sustainable development goals.

Intended Uses

Given the nature of the training data, the CityPulse model is best suited for detecting physical alterations in the built environment. The model accepts two 224x224 RGB images as input. For urban change detection tasks, the input images should be taken at roughly the same location but at different times. The output of the model is a value ranging from 0 to 1, which can be interpreted as the model's inference of the probability of changes occurring in the buildings depicted in the images.

Sample Code

The example Python code for inference using this model checkpoint is shown in the Github Repo.

Limitations

Although has been tested on a dataset comprising over 12,700 image pairs, the model robustness remains unclear when faced with noisier images, more complex urban environments, and more diverse urban styles.

Training

Model

• Architecture: a Siamese DINOv2 model with ViT-B as the backbone

• Patch size: 14

• ViT feature dim: 768

Dataset

Our curated CityPulse dataset comprises 25,423 image pairs, including 4,465 images across five US cities, with a timespan exceeding 16 years.

Optimize

• Optimizer: Adam

• Learning rate: 1e-5

• Gradient clip: 0.5

• GPUs: 4 x Nvidia Tesla T4

Software

• PyTorch

• Pytorch-Lightning

Environment

Please refer to citypulse.yaml in the Github Repository for environment configuration.

License

The model is licensed under the MIT license.

Citing

If you found this model checkpoint useful, please consider citing:

@article{huang2024citypulse,
  title={CityPulse: Fine-Grained Assessment of Urban Change with Street View Time Series},
  author={Huang, Tianyuan and Wu, Zejia and Wu, Jiajun and Hwang, Jackelyn and Rajagopal, Ram},
  journal={arXiv preprint arXiv:2401.01107},
  year={2024}
}