Ornith-1.0-35B-Vision-NVFP4-Calibrated

Ornith-1.0-35B — NVFP4, calibrated, with vision

NVFP4 quantization of ornith-ai/Ornith-1.0-35B (Qwen3.5-MoE architecture, multimodal), built with NVIDIA ModelOpt and — unlike the existing community NVFP4 — calibrated on real data. Vision works: the full vision tower ships at BF16 and is verified below on a real image test (awq_clip, 256 samples: 192 UltraChat-200k conversations + 64 real Python source files). Serves directly in vLLM on Blackwell (built and verified on a GB10, sm_121).

vllm serve kingjones777/Ornith-1.0-35B-Vision-NVFP4-Calibrated --max-model-len 8192

Measured — not estimated

A/B against sakamakismile/Ornith-1.0-35B-NVFP4 (the 378K-download dataless build), run sequentially on the same idle GB10, same vLLM build, same flags:

Honest summary: speed parity, identical gate results. The difference is the recipe — this build's non-expert Linear layers had their quantization clipping calibrated against real activations; the community build is dataless (memoryless_minmax, verified from its shipped recipe.yaml). Whether calibration shows up in your workload is for your evaluation to decide; no stronger claim is made here.

Recipe transparency

• Quantized: all Linear layers except the vision tower (27 blocks, kept BF16), MoE router gates (kept BF16), and lm_head (kept BF16) — same protection set as the community build.
• embed_tokens is nn.Embedding and is BF16 in both builds (NVFP4 recipes target Linear only — an ignore-list entry for it would be a no-op).
• ⚠️ The 30,720 fused-expert projections are quantized dataless in both builds — ModelOpt's AWQ path cannot reach the transformers-5.x fused expert container. Calibration here covers the 310 non-expert Linears.
• An awq_lite variant was built and rejected: it exports pre_quant_scale tensors that vLLM silently ignores, which would serve incorrect math while appearing to work.

Verification

Every gate above was run against the served model over the OpenAI API, answers asserted against content + reasoning with finish_reason recorded. File sizes on this repo are byte-verified against the build box. Built on NVIDIA GB10 (Blackwell), CUDA 13.0, ModelOpt 0.44, vLLM nightly cu130.

Credits

Base model by ornith-ai. Quantized with NVIDIA ModelOpt. The community NVFP4 by sakamakismile was the benchmark target and its disclosed recipe made the controlled comparison possible.