OmniLion-NVFP4-W4A16

OmniLion NVFP4 W4A16

This is the weight-compressed serving derivative of OmniLion revision ef46e7303b1770a46760effebfd220ba0fc4ceab. It combines the SEA-LION/Qwen3.6 language and vision model, Qwen3-ASR audio encoder, and trained P21 audio projector in one SafeTensors checkpoint.

• Weights: packed NVFP4, group size 16
• Activations: BF16
• Verified GB10 backend: vLLM MarlinNvFp4LinearKernel
• Calibration: none; ModelOpt weight-only W4A16 requires no calibration pass

The visual tower, audio tower, projector, MTP payload, LM head, embeddings, and fused linear-attention projections remain unquantized. Supported inputs are text, image, video, audio up to 30 seconds, and joint video plus audio in either content-part order.

This is not GGUF. Install the bundled omnilion-vllm-plugin wheel before starting vLLM.

Serve

python -m pip install --no-deps ./omnilion_vllm_plugin-0.1.0-py3-none-any.whl
export VLLM_PLUGINS=omnilion
export PYTHONNOUSERSITE=1

vllm serve LLJYY/OmniLion-NVFP4-W4A16 \
  --served-model-name OmniLion \
  --host 127.0.0.1 \
  --port 8002 \
  --dtype bfloat16 \
  --max-model-len 8192 \
  --max-num-seqs 2 \
  --gpu-memory-utilization 0.65 \
  --limit-mm-per-prompt '{"image":1,"video":1,"audio":1}' \
  --media-io-kwargs '{"video":{"num_frames":30}}' \
  --chat-template-content-format string \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --enforce-eager

Pin the immutable repository commit in production.

Verification and explicit overrides

Structural verification passed: 304 packed NVFP4 weights, 304 first-level scales, 304 second-level scales, zero activation scales, and 840 intentionally unquantized tensors bit-exact with the immutable BF16 parent. Native vLLM checks passed for text, image, video, audio, both joint audiovisual orders, oversized-audio rejection, normal thinking, and authentication. A fixed multilingual text panel scored 8/8.

The derivative was promoted by explicit owner override, not by changing failed gates into passes:

• The BF16 parent itself retains evaluation_gate_passed: false and was promoted by its separate owner waiver.
• W4A16 changed one fixed audio multiple-choice answer from BF16's correct C (teal blanket) to A (peach blanket).
• Therefore audio_exact_bf16_runtime_match: false and automatic_runtime_gate_passed: false remain disclosed.
• Publication and serving were separately authorized because a deployable NVFP4 artifact was required under the event deadline.

Known BF16-parent proxy measurements also remain disclosed: Tamil-900 non-inferiority failed, throat clearing scored 7/8, and the old exact-empty-plus-EOS incomplete-speech proxy scored 0/32. That incomplete-speech test is not a settled product objective.

These are bounded release canaries, not comprehensive capability guarantees. Native W4A4 activation quantization is not claimed. Training data, calibration data, and evaluation media are not included.

See BENCHMARKS.md, PROVENANCE.md, LICENSES.md, and release-manifest.json.