geoguesser-qwen3.5-4b-grpo

GeoGuesser · Qwen3.5-4B · GRPO run 1

A LoRA adapter that plays GeoGuessr: dropped at a random street corner on Earth, it works out where it is and commits to a coordinate. Trained with GRPO against a live OpenEnv environment serving real Mapillary panoramas.

This is run 1, step 1000, and it is the adapter behind every number in the write-up.

What it scores

200 held-out tasks across 73 countries, pass@4, 800 episodes per arm, all scored through the same curve. The full board is in the project README.

Against its own base, paired per task: +0.1620 ± 0.0137 (95% CI), better on 169 of 200 tasks. Country identified 21.2% → 33.1%. Episodes floored at zero on the environment's own curve 29.5% → 0.5%.

Scores are on the training reward curve, min(1, 0.5·e^(-d/1492.7) + 0.5·e^(-d/5000)) × (1 - min(cost, 0.2)), recomputed from raw distance. That is not GeoGuessr's own curve, which floors at zero past roughly 3,500 km.

How it behaves, which is the interesting part

It stopped exploring. Turns per episode fell from 6.7 to 1.1, output tokens from 1,062 to 66, and accuracy went up. It recognises the region from the first frame and commits to a plausible city in it, rather than reading signs across a dozen turns. That is not a random guess (a random guess on land averages roughly 8,000 km of error; this is 662 km) and it is not reward hacking: no identity leak in 882 audited prompts, no repeated tasks, and an action cost of 0.002 on the converged policy. It is what the reward rewarded, and the write-up is about why.

Training

Note that it plateaued by step 200 (0.6393 against step 1000's 0.6445, inside the noise). If you reproduce this, stop at 250.

Using it

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-4B", device_map="auto")
model = PeftModel.from_pretrained(model, "HuggingEnvs/geoguesser-qwen3.5-4b-grpo")
processor = AutoProcessor.from_pretrained("Qwen/Qwen3.5-4B")

Or serve it with vLLM and point the eval harness at it:

vllm serve Qwen/Qwen3.5-4B --enable-lora \
  --lora-modules run1=HuggingEnvs/geoguesser-qwen3.5-4b-grpo

Everything else

• The write-up, which explains every decision and what it cost to learn it
• The environment, playable in a browser
• The task splits and the imagery
• All four training runs in one dashboard
• Code and exact commands

Imagery is Mapillary, CC BY-SA 4.0.

Citation

@misc{fineenvs,
  author = {Kolavi, Adithya S},
  title  = {FineEnvs: Open Source RL Environments for LLM Agents},
  year   = {2026},
  url    = {https://github.com/adithya-s-k/FineEnvs}
}