GeoSynth-ControlNets-Location

[!WARNING] we do not have a full checkpoint conversion validation, if you encounter pipeline loading failure and unsidered output, please contact me via [email protected]

GeoSynth-ControlNets-Location

Repository for location-conditioned GeoSynth ControlNets.

Location (lon/lat) conditioning is the primary workflow for geo-aware synthesis. Default: St. Louis, MO (lon=-90.2, lat=38.6). Use SatCLIP + CoordNet for full diffusers-style location conditioning.

We maintain two repositories—one per base checkpoint—each with its compatible ControlNets:

GeoSynth-Location-SAM controlnet ckpt is missing from source.

This repository

1. GeoSynth-Location base — Converted from geosynth_sd_loc-v3.ckpt to diffusers format. Text encoder and UNet are the same as SD 2.1 (not fine-tuned). The original checkpoint also includes a CoordNet branch for [lon, lat] conditioning (see Architecture).
2. ControlNet models — GeoSynth-Location-OSM, GeoSynth-Location-Canny (converted from SD-style checkpoints under MVRL/GeoSynth-Location-OSM and MVRL/GeoSynth-Location-Canny), and GeoSynth-Location-SAM, located under controlnet/.

Architecture

The full location pipeline adds a CoordNet branch to the base LDM:

• Input: [lon, lat] → SatCLIP location encoder → CoordNet (13 stacked cross-attention blocks, inner dim 256, 4 heads) → conditioning injected into UNet
• ControlNet and CoordNet jointly condition the UNet (see GeoSynth paper Figure 3)

ControlNet variants (this repo)

Model Sources

• Source: GeoSynth
• Paper: GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis
• Base model: Stable Diffusion 2.1
• Related: GeoSynth-ControlNets (non-location models)

Usage

CLI:

python inference_demo.py --control demo_images/GeoSynth-Location-OSM/input.jpeg --control_type OSM --lon -90.2 --lat 38.6

Python:

import sys, os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

import torch
from PIL import Image
from geosynth_pipeline import load_geosynth_pipeline_with_location, run_with_location

pipe = load_geosynth_pipeline_with_location(".", controlnet_subfolder="controlnet/GeoSynth-Location-OSM", local_files_only=True)
pipe = pipe.to("cuda")

img = Image.open("demo_images/GeoSynth-Location-OSM/input.jpeg").convert("RGB").resize((512, 512))
output = run_with_location(pipe, "Satellite image features a city neighborhood", image=img, lon=-90.2, lat=38.6)
output.images[0].save("generated_city.jpg")

Citation

If you use this model, please cite the GeoSynth paper. For location-conditioned variants, also cite SatCLIP.

@inproceedings{sastry2024geosynth,
  title={GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis},
  author={Sastry, Srikumar and Khanal, Subash and Dhakal, Aayush and Jacobs, Nathan},
  booktitle={IEEE/ISPRS Workshop: Large Scale Computer Vision for Remote Sensing (EARTHVISION)},
  year={2024}
}

@article{klemmer2025satclip,
  title={{SatCLIP}: {Global}, General-Purpose Location Embeddings with Satellite Imagery},
  author={Klemmer, Konstantin and Rolf, Esther and Robinson, Caleb and Mackey, Lester and Ru{\ss}wurm, Marc},
  journal={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={39},
  number={4},
  pages={4347--4355},
  year={2025},
  doi={10.1609/aaai.v39i4.32457}
}