smolvla-base-onnx-fp16

SmolVLA base — ONNX, fp16-compressed for in-browser WebGPU

A fp16-weight-compressed ONNX build of SmolVLA (HuggingFace's 450M vision-language-action robot policy), packaged to run fully in a web browser on WebGPU via onnxruntime-web. Derived from the componentized export ainekko/smolvla_base_onnx; large weight tensors are stored as fp16 and cast back to fp32 at use, so compute stays fp32-accurate (action output matches the fp32 reference to ~2.5e-4) while the download shrinks 1.58 GB → 831 MB.

Used by the Institute for Physical AI to run the real frontier VLA live in the browser — no server, no install.

Components (opset 17, static shapes)

smolvlm_vision (image [1,3,512,512] → [1,64,960]) · smolvlm_text (token embeddings) · smolvlm_expert_prefill / smolvlm_expert_decode (VLM + action expert, 16 KV layers) · state_projector · action_in/out_projector · time_in/out_projector.

Inference

Flow-matching: embed_prefix (vision·√d ‖ text·√d ‖ state, padded to 177) → prefill (16-layer KV cache) → 10-step Euler denoise (action_in ⊕ sinusoidal-time → time MLP → decode with KV → action_out = velocity) → action chunk [50, 32]. Reference orchestrator: robobrowser/smolvla.js.

Preprocessing: resize 512×512, RGB, normalize (x-0.5)/0.5 → [-1,1]; SmolVLM2 tokenizer.

License Apache-2.0, following SmolVLA.