Downloads · 30 days
0
astonishingwolf/blorptron-qzx9-vla-scratchpile-7k2f
blorptron-qzx9-vla-scratchpile-7k2f is a machine learning model from astonishingwolf. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
A research snapshot on RL fine-tuning and test-time correction of OAT-tokenized autoregressive (AR) policies for robomimic (square, toolhang, state obs, abs10 rot6d actions). Full code, all checkpoints, all tokenizers…
Downloads · 30 days
0
Access
Public
Updated Sep 8, 2026
Repo size
14.1 GB
Likes
0
Public
Click a slice to open those files.
.pt8.4 GB · 58%
From the Hugging Face model README
A research snapshot on RL fine-tuning and test-time correction of OAT-tokenized autoregressive
(AR) policies for robomimic (square, tool_hang, state obs, abs10 rot6d actions). Full code,
all checkpoints, all tokenizers, and every experiment record are included so results are reproducible
and browsable.
⚠️ Research artifact, not a polished library. Two codebases (
much-ado-about-noising,oat) plusexperiments/with the running lab notebook in eachPLAN.md.
The ~0.75 success ceiling on square was not an RL problem — it was quantization-induced
off-manifold drift. The winning levers were test-time manifold projection (a denoiser), a
finer tokenizer, HL-Gauss label smoothing, and dropping mixed-quality (MH) demos.
Train a state-conditional action denoiser D(a,s) by denoising score matching (OU corruption). At
rollout, project each policy action a → a + α·(D(a,s) − a) before executing — snapping quantization
drift back onto the valid-action manifold. No RL, pure test-time. Frozen 128-ep:
| Task (8-token BC) | plain (α=0) | + projection (α=1) | gain |
|---|---|---|---|
| Square | 0.688 | 0.867 | +18 |
| Tool Hang | 0.383 | 0.633 | +25 |
Controlled: feeding the denoiser a null action → 0.000, so it refines the policy's action, not replaces it (rules out "denoiser is just a better policy").
A 16-register, 4096-vocab, no-dropout tokenizer (reconstruction MSE 7e-5, ~10× finer than the 8-register one) + HL-Gauss label smoothing (per-FSQ-dim Gaussian soft targets) + PH-only square data. Frozen 128-ep on the final best BC checkpoints:
| Task (16-token BC) | plain (α=0) | + projection (α=1) | old 8-tok projected |
|---|---|---|---|
| Square | 0.883 | 0.922 ✅ (>90%) | 0.867 |
| Tool Hang | 0.578 | 0.695 | 0.633 |
Square BC alone (no denoising) already reached 0.883 — the finer tokenizer + HL-Gauss did most of
the work; denoising added the last +3.9 to clear 90%. Best checkpoints: square seed302_success96
(0.96 in-train), tool_hang seed301_success74 (0.74 in-train).
Mechanism confirmed: a finer tokenizer raises the baseline a lot (sq .69→.81, th .38→.54) but shrinks the denoiser's marginal gain (sq +18→+8, th +25→+15) — tokenizer resolution and denoising are partly substitutes (both fix quantization drift).
square down (~0.58 plateau); PH-only climbed to 0.90 in-training.Per-FSQ-dim Gaussian soft targets (network.hl_gauss) beat hard cross-entropy at matched steps on
both tasks, and let tool_hang reach its old 300k plateau ~10× faster.
much-ado-about-noising/ # policy/RL framework (MIP two-step flow + OAT-AR)
mip/networks/ar_decoder.py # AR token decoder (+ HL-Gauss ce_loss)
mip/losses.py # oat_ar (BC), awsi/awsi_v3 (V1 softmax-SI, V2 PPO, CISPO, GSPO), energy reward
mip/energy_reward.py # denoiser + manifold-energy / projection helpers
examples/train_robomimic.py # BC + RL entrypoint
examples/train_action_denoiser.py # trains the projection denoiser (DSM, OU noise)
examples/projection_eval.py # test-time manifold projection eval (α sweep)
checkpoints/ # 62 model checkpoints (BC, RL, denoisers)
oat/ # OAT action tokenizer (FSQ + register encoder)
output/.../checkpoints/latest.ckpt # 59 tokenizers incl. 16-tok 4k-vocab
experiments/ # lab notebooks: each PLAN.md is the running record
exp_20260905_action_energy_reward/PLAN.md # projection + HL-Gauss + 16-tok program
exp_20260830_robomimic_lift_repro/PLAN.md # RL surrogate campaign + diagnostics + plots/
much-ado-about-noising/checkpoints/action_denoiser_{square,toolhang}.ptoat/output/20260905/194350_train_oattok_mt5phmh_abs10_N1100/checkpoints/latest.ckpt...square_ph_state_oat_ar_..._seed302_success90.pt...tool_hang_ph_state_oat_ar_..._seed301_success56.ptcd much-ado-about-noising
PROJ_TASK=square_ph_state \
PROJ_CKPT=checkpoints/square_ph_state_oat_ar_ar_decoder_256_seed302_success90.pt \
PROJ_TOK=<oat 16-tok latest.ckpt> \
PROJ_DENOISER=checkpoints/action_denoiser_square.pt \
PROJ_ALPHA=1.0 python examples/projection_eval.py # α=0 baseline vs α=1 denoised, frozen 128-ep
(Excluded from this snapshot: .venv, raw robomimic datasets (redownloadable from HF), wandb runs.)