Downloads · 30 days
0
Aasdfip/longnav-objectnav-flow-nopose-cotrain-2p5hz-rl-a09-ck303
longnav-objectnav-flow-nopose-cotrain-2p5hz-rl-a09-ck303 is a reinforcement learning model from Aasdfip. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
RL fine-tune of longnav-objectnav-flow-nopose-cotrain-2p5hz, by flow-SDE policy gradient on continuous-control ObjectNav. Checkpoint 303 of the flowsdefreezeheada09 run.
Downloads · 30 days
0
Access
Public
Updated Aug 15, 2026
Repo size
588 MB
Likes
0
Public
Click a slice to open those files.
.safetensors558 MB · 94%
From the Hugging Face model README
RL fine-tune of longnav-objectnav-flow-nopose-cotrain-2p5hz,
by flow-SDE policy gradient on continuous-control ObjectNav. Checkpoint 303 of the
flow_sde_freezehead_a09 run.
The base is the MERGED repo, not raw Qwen. This adapter holds the RL delta alone
(~2% of the SFT adapter's magnitude). Both turn_vector_head_config.json (model.model_id
-- this is the field the flow_rollout backend actually reads) and adapter/adapter_config.json
point at Aasdfip/longnav-objectnav-flow-nopose-cotrain-2p5hz-merged, so --ckpt <this repo>
works with the same code that ran the SFT model, no changes. Applying it to raw
Qwen/Qwen3-VL-2B-Instruct instead would evaluate the pretrained VLM plus a 2%
perturbation — near-zero scores, and no error anywhere.
Same 400 HM3D val episodes, same packing and seeds, --no-pose-injection on both arms:
| metric | this checkpoint | SFT baseline | delta |
|---|---|---|---|
| oracle success | 0.695 | 0.650 | +0.045 |
| oracle SPL | 0.500 | 0.401 | +0.099 |
| success | 0.554 | 0.554 | 0.000 |
| SPL | 0.272 | 0.254 | +0.018 |
| soft SPL | 0.316 | 0.291 | +0.025 |
McNemar on oracle success: 46 episodes fixed, 28 broken, z = 1.98, p = 0.048 — significant, but marginally so.
Read it carefully: success is identical (0.554 both). success requires the harness's
auto-stop heuristic to fire at the right moment; the oracle metrics credit ever reaching the
goal. So this policy reaches more goals and reaches them far more efficiently (oracle SPL
+25% relative) while converting none of that into better stopping — it has no stop head
and was trained with no success bonus.
| algorithm | flow-SDE policy gradient (PPO-style) on the denoising chain, N=3 of K=10 steps stochastic |
| exploration | a = 0.9, chosen from a measured noise ladder (competence is flat 0.15-0.7, shoulders at 1.0, breaks by 1.5) |
| action head | FROZEN (action_head_learning_rate = 0); gradients flow through it into the backbone LoRA |
| base | SFT LoRA merged into the weights; the trainable LoRA starts as a zero delta |
| tether | KL to the SFT policy measured, not constrained (disable_adapter() is an exact reference once the base is merged) |
| training pool | 24 episodes over 4 HM3D train scenes — deliberately tiny, an overfit diagnostic |
| duration | ~300 cycles at lr 2e-6 |
The training pool is scene-disjoint from HM3D val, so the sample400 numbers above are generalisation from 24 episodes, not memorisation. What plausibly transferred is execution (approach, path efficiency), which is consistent with oracle SPL moving most.
| file | what |
|---|---|
adapter/ | LoRA (r=128, alpha=256) — the RL delta only, over the merged base |
turn_vector_head.pt | flow-matching action head; frozen during RL, so this is the SFT head at fp32 (the RL checkpoint's own copy is a bf16 round-trip of it, 3.9e-3 apart) |
turn_vector_head_config.json | head config |
| tokenizer / preprocessor files | as trained |
Identical to the SFT model's command with --ckpt pointed here:
python scripts/eval_objectnav_policy.py \
--episodes <hm3d objectnav val split> \
--scene-root <scene_datasets> \
--output-dir <out> \
--episode-ids "$(cat sample400_ids.txt)" \
--ckpt <this repo> \
--policy-python <env with transformers> \
--policy-sys-path <longnav src> \
--policy-backend flow_rollout --policy-seed 1000 \
--max-steps 175 --dt 0.04 --gap 10 \
--success-distance 1.0 --distance-to VIEW_POINTS \
--seed 0 --navmesh dataset --fresh-sim-per-episode \
--auto-stop --auto-stop-delay 15 \
--no-pose-injection \
--pid-preset baseline
--no-pose-injection matters: this policy family scores better with the pose feed off
(0.663 vs 0.614 oracle on sample101), and the RL run's conversations were the nopose format.
Running this adapter on the merged base is closer to training conditions than the numbers
in the table above. Our own evaluation composed the SFT and RL LoRAs into one
rank-stacked adapter over the raw base, which never materialises the bf16-rounded SFT sum
that the RL run actually trained against. The merged repo is bitwise identical to the
training base (verified torch.equal, max diff 0.0, across four probe modules); the stacked
path differs from it by that rounding — 0.11 Frobenius on q_proj, ~5x the RL delta's own
norm, though unstructured and ~0.17% of the weight norm.
How much that moves a metric is only anecdotally known. On two episodes run through both paths with identical seeds and packing: a 3.4 m episode came out nearly identical (final 0.33 m vs 0.47 m, SPL 0.370 vs 0.371), while a 22.1 m episode diverged completely -- the merged path reached the goal (0.18 m, SPL 0.815) and the stacked path stalled at 2.58 m closest approach and failed. A small base perturbation is negligible over a short horizon and can be decisive over a long one. Expect differences from the table, in either direction; they are not a reproduction failure.
Per-episode seeds are also positional (seed + position in the run), so a differently-sized
or differently-ordered episode set reseeds every episode — roughly 40% of episodes flip
outcome between identical passes of this policy family. Reproduce the table with the full
400-episode list in the given order, not a subset.