GR00T-N1.7-G1-SONIC-LAFAN

GR00T-N1.7 × GEAR-SONIC — LAFAN whole-body motion for Unitree G1 (flow3)

⚠️ Proof-of-concept (derisk checkpoint) — read first

This model memorizes 8 hand-picked LAFAN windows, 1 episode each. Generalization to new prompts/motions is untested (train-set reconstruction MSE ≈ 0.0026, i.e. memorization, no held-out split). The demo clips below are driven with the tracking deviation/fall terminations disabled so each window plays in full — they show prompt → token tracking, and "physically valid" (never fell) is not the same as "visually faithful". Not a deployable general VLA.

A single GR00T N1.7 vision-language-action model fine-tuned to emit the 64-dim FSQ motion token of the GEAR-SONIC whole-body controller (WBC). One model, one prompt → the frozen SONIC WBC decodes a full-body G1 motion (it supplies balance/recovery). No per-joint behavior cloning.

prompt + ego-cam + proprio → GR00T N1.7 → 64-d token → SONIC.decode(50 Hz) → 29-DoF G1

Variant — LAFAN: trained on wsagi/SONIC-VLA-LAFAN, whose motions are hand-picked windows from the LAFAN1 corpus (Ubisoft). Its sibling wsagi/GR00T-N1.7-G1-SONIC-BonesSeed uses NVIDIA's GEAR-SONIC demo motions — same prompt → token → WBC pipeline, different source.

🔗 Code, training & validation scaffolding: vitorcen/LeSONIC — the SONIC VLA project (full principle + dataset pipeline in doc/sonic_vla_principles.html), part of the vitorcen/isaaclab-experience umbrella.

---

🎥 flow3 — looping demo (main)

fight → run → fight → run → dance → run, looping in one session. The GR00T-predicted tokens of six windows are concatenated and decoded by the WBC (offline playback = guaranteed smooth).

<video controls src="https://huggingface.co/wsagi/GR00T-N1.7-G1-SONIC-LAFAN/resolve/main/videos/flow3.mp4"></video>

🎥 Per-window closed-loop demos

Fight — combat strikes and combo kicks · block and push-kick · fierce swings <video controls src="https://huggingface.co/wsagi/GR00T-N1.7-G1-SONIC-LAFAN/resolve/main/videos/fight1.mp4"></video> <video controls src="https://huggingface.co/wsagi/GR00T-N1.7-G1-SONIC-LAFAN/resolve/main/videos/fight2.mp4"></video> <video controls src="https://huggingface.co/wsagi/GR00T-N1.7-G1-SONIC-LAFAN/resolve/main/videos/fight3.mp4"></video>

Run — jog forward then run backward · sprint back and forth then backpedal · run in a circle <video controls src="https://huggingface.co/wsagi/GR00T-N1.7-G1-SONIC-LAFAN/resolve/main/videos/run1.mp4"></video> <video controls src="https://huggingface.co/wsagi/GR00T-N1.7-G1-SONIC-LAFAN/resolve/main/videos/run2.mp4"></video> <video controls src="https://huggingface.co/wsagi/GR00T-N1.7-G1-SONIC-LAFAN/resolve/main/videos/run3.mp4"></video>

Dance — moonwalk · spin, step back, and clap <video controls src="https://huggingface.co/wsagi/GR00T-N1.7-G1-SONIC-LAFAN/resolve/main/videos/dance1.mp4"></video> <video controls src="https://huggingface.co/wsagi/GR00T-N1.7-G1-SONIC-LAFAN/resolve/main/videos/dance2.mp4"></video>

---

How LAFAN motions get in: a physical-validity gate

LAFAN clips are foreign choreography to the frozen WBC, so a clip can't simply be "trained on". Each candidate window is first screened by letting the frozen WBC physically track it and asking "did the robot actually fall?" (root height + torso tilt) — not the release's strict per-frame safety envelope, which terminates on a single fast-strike frame even when the robot never falls. Of 10 hand-picked windows, 2 pass the strict gate but 6 never fall; those 6 (plus 2 trackable prefixes) are what this model is trained on. This is a WBC-capability screen, upstream of the VLA: a token is only worth learning if the WBC can physically realize it.

Contents

▶️ How to run

The model emits SONIC tokens; you need the GEAR-SONIC WBC to decode them. See vitorcen/LeSONIC:

# offline (smooth, no server):
bash scripts/gear_sonic_flow3.sh
# live closed-loop (GR00T server in the loop):
GR00T_CKPT=<this checkpoint> bash scripts/gear_sonic_live_demo.sh @flow3

⚠️ Honest scope & limitations

• Memorization, not generalization — 8 windows × 1 ep; no held-out. MSE ≈ 0.0026 is train-set.
• "Never fell" ≠ "looks like the action" — visual fidelity is filtered separately in GUI; some windows (e.g. moonwalk) track loosely. The 2 dropped fight windows fell immediately (bad start pose).
• Demos run with deviation terminations disabled so the full window plays; not a balance/fall test.
• Single-frame obs, no memory — one-shot windows may settle toward standing under live inference.
• Depends on the external SONIC WBC + (for live) a ZMQ server.

Provenance / lineage

LAFAN1 (Ubisoft) → retarget to G1 (MimicKit) → physical-validity screen
   → nvidia/GEAR-SONIC WBC tracks each window, record tokens → wsagi/SONIC-VLA-LAFAN
   → finetune nvidia/GR00T-N1.7-3B → this model

Please cite GEAR-SONIC, BONES-SEED (the WBC's training corpus), and LAFAN1. Use under the NVIDIA Open Model License terms of the upstream release; respect Ubisoft's LAFAN1 license.