cosmos_nano_policy_so101

cosmos_nano_policy_so101

A 6-DOF SO-101 bin-placement action policy, post-trained from nvidia/Cosmos3-Nano on one RTX PRO 6000 (96 GB). This is checkpoint iteration 3750 of the run, the only checkpoint that has completed the task in evaluation. The LoRA adapters are merged into the base weights, and the checkpoint is exported as consolidated safetensors.

It is a research checkpoint with a low success rate. Read Benchmark before using it.

Training

Benchmark

Evaluated in the so101_bench Isaac Lab digital twin, So101Bench-Bin-v0: 100 single-object episodes (tasks/focus5.jsonl), 25 s per episode, 32 actions executed per inference call.

• Every success had the object on the same side of the table as the bin. The far-side half of the task is unsolved.
• Nearly all failures are time-outs in which the target object never leaves the table. The policy either grasps and places the object, or never lifts it.
• Checkpoints 3500 and 4000 each scored 0 / 20. Iteration 4000 has the lowest training loss, so training loss did not predict success.
• The results depend on the action horizon: horizon 32 scored 4 / 70 while horizon 16 scored 0 / 63 on the same server and scenes.

Serving

The checkpoint is served with cosmos_framework.scripts.action_policy_server_robolab, an openpi websocket server. Every SO-101 flag below is required, and a missing one fails silently: the server starts but returns wrong actions.

huggingface-cli download kabilanKB/cosmos_nano_policy_so101 --local-dir cosmos_nano_policy_so101

python -m cosmos_framework.scripts.action_policy_server_robolab \
    --checkpoint-path cosmos_nano_policy_so101 \
    --port 8000 \
    --domain-name so101 \
    --action-dim 6 \
    --arm-joint-dim 5 \
    --action-space joint_pos \
    --conditioning-fps 30 \
    --no-flip-gripper \
    --action-normalization minmax \
    --normalizer-stats-path cosmos_nano_policy_so101/so101_lerobot_stats.json \
    --view-description 'The top half is from the front-facing wrist camera. The bottom half is from the fixed overhead camera.' \
    --no-guardrails

These flags depend on SO-101 support in the policy server (--arm-joint-dim, --no-flip-gripper, --action-normalization, --view-description), which is not in upstream cosmos-framework 5e67049. The pipeline that runs training, merge, export, serving and evaluation is in kabilankb/so101-cosmos-nano-policy.

Request format: prompt (the instruction), observation/image (the 960×640 concatenated view), observation/joint_position (5 values), observation/gripper_position (1 value).

Response: action, 32 × 6 absolute joint targets in raw .pos units.

Limitations

• Trained and evaluated only in simulation. It has not been tested on a physical SO-101.
• Covers five objects and one task family, with 18–25 demonstrations per instruction.
• Sampling is stochastic: the server draws a new seed for every request, and no scene has succeeded reliably.

License and attribution

This model is a derivative of NVIDIA Cosmos3-Nano, released under the OpenMDW License 1.1, and is distributed under the same license. Cosmos3-Nano's language backbone derives from Qwen/Qwen3-VL-8B-Instruct (Apache 2.0), whose tokenizer and processor files are bundled here.

Built with NVIDIA Cosmos. Post-trained by Kabilan KB.