streetview-eu-geocell-heads

Street View Europe — Geocell Classification Heads

Trained linear classification heads for European Street View geolocation, sitting on top of a frozen geolocal/StreetCLIP vision encoder (1024-dim pooled features).

Each head predicts one of N discrete geocells (a partition of Europe into bounded regions). The cell centroids ship alongside each head as geocell_info.json, so a prediction can be turned into a (lat, lon) by indexing into the file.

These checkpoints power the live demo at lebfla11/streetview-eu-geocell-demo.

---

Quick start

import json, torch, torch.nn as nn
from huggingface_hub import hf_hub_download
from transformers import CLIPImageProcessor, CLIPModel
from PIL import Image

REPO = "lebfla11/streetview-eu-geocell-heads"

# 1. Pull a head + its partition (★ headline: v13 ce_haversine λ=0.05)
head_path = hf_hub_download(REPO, "ablation_v13_weighted/lam_0.05/best.pt")
info_path = hf_hub_download(REPO,
    "data_collection/outputs/geocells_h3_res4_eu/geocell_info.json")

centroids = {int(k): (v["centroid_lat"], v["centroid_lon"])
             for k, v in json.load(open(info_path)).items()}
n_cells = len(centroids)

# 2. Build the head (matches HeadOnlyModel(feat_dim=1024, fusion='feature'))
class Head(nn.Module):
    def __init__(self, n_cells, dropout=0.3):
        super().__init__()
        self.head = nn.Sequential(
            nn.LayerNorm(1024), nn.Dropout(dropout), nn.Linear(1024, n_cells))
    def forward(self, feats):
        # feats: (B, V, 1024) for V views, mean-pool then head
        if feats.dim() == 3:
            feats = feats.mean(dim=1)
        return self.head(feats)

head = Head(n_cells).eval()
ck = torch.load(head_path, map_location="cpu", weights_only=False)
head.load_state_dict(ck["model"] if isinstance(ck, dict) and "model" in ck else ck,
                     strict=False)

# 3. Encode an image with frozen StreetCLIP
clip = CLIPModel.from_pretrained("geolocal/StreetCLIP").vision_model.eval()
proc = CLIPImageProcessor.from_pretrained("geolocal/StreetCLIP")
img  = Image.open("street_view.jpg").convert("RGB")
px   = proc(images=img, return_tensors="pt")["pixel_values"]
with torch.no_grad():
    feat = clip(pixel_values=px).pooler_output.float()  # (1, 1024)
    logits = head(feat)
    probs = logits.softmax(-1)

top5 = probs[0].topk(5)
for prob, cell in zip(top5.values, top5.indices):
    lat, lon = centroids[int(cell)]
    print(f"cell {int(cell):4d}  {prob.item()*100:5.1f}%  ({lat:.3f}, {lon:.3f})")

For a head trained with fusion='logit' (the v12 logit_lr5e-3 family — not shipped here, see the demo Space for context), apply the linear head per view and average the logits instead of the features.

---

Model details

Available heads

All checkpoints were trained on the leakage-free train/val/test split (no shared lat,lon key between splits). Earlier runs/clusters/* runs that used a leaky split are not included here.

For each head, the matching partition lives at data_collection/outputs/geocells_<name>/geocell_info.json (centroids only — the training-time geocell_map.json and splits.json are not shipped).

Training data

• Source: Internal Google Street View collection
• Size: 79 144 European locations × ~4 headings = 316 498 images
• Coverage: 41 European countries
• Split: 80 / 10 / 10 train/val/test, stratified to be leakage-free (no shared lat,lon key between splits)

Evaluation (test split, headline head: ablation_v13_weighted/lam_0.05)

The v13 head is a 10-epoch warm-started fine-tune from ablation_v12/seed_44/best.pt with the new ce_haversine_weighted loss (L = CE + λ · mean_haversine(posterior-expected lat/lon, true GPS), λ=0.05). The fine-tune additionally improves cross-domain generalization on out-of-distribution test sets (im2gps Europe top-1 14.4 → 20.0; OSM-Europe-1k 4-view mean km 716 → 629).

Intended use

• Educational / demo: explore CLIP-based classifier behaviour under different geocell granularities.
• Reference implementation: starting point for further geolocation research using the StreetCLIP vision backbone.
• Thesis demonstrator: companion artefact for the master's thesis at FH JOANNEUM University of Applied Sciences.

Out-of-scope / limitations

• Geographic scope: Europe only. Predictions outside Europe are meaningless extrapolations.
• Visual scope: Street View-style imagery (driver perspective, outdoor, daylight). Aerial photos, indoor shots, screenshots, and composite imagery are likely to fail.
• Resolution ceiling: predictions are cell centroids; per-cell median errors range from ~16 km (k=4000 EU) to ~120 km (k=500 EU).
• Class imbalance: dataset is skewed toward Western/Central Europe. Predictions for under-represented regions carry higher uncertainty.
• Privacy: do not use this model to identify or track individuals.

Citation

@misc{streetclip2023,
  title  = {StreetCLIP: A Robust Image-Language Model for Generalizable Geolocation},
  author = {Lukas Haas and Silas Alberti and Michal Skreta},
  year   = {2023},
  url    = {https://huggingface.co/geolocal/StreetCLIP}
}

@mastersthesis{leber2026streetview,
  title  = {Street View Image Geolocation in Europe via Geocell Classification},
  author = {Florian Leber},
  school = {FH JOANNEUM University of Applied Sciences},
  year   = {2026}
}

Contact

Florian Leber — [email protected] Master's thesis at FH JOANNEUM University of Applied Sciences. Source: https://git-iit.fh-joanneum.at/leberflo19/europestreetviewgeolocator

License

MIT. Released for research / educational use.