Tessera-4B-Preview-NVFP4A16

Tessera-4B-Preview — NVFP4A16

4-bit NVFP4 (weight-only) quantization of sahilchachra/Tessera-4B-Preview, a Qwen3.5-4B reasoning / agentic vision-language model with a <think> reasoning channel and tool / function-calling support.

This variant targets NVIDIA Blackwell GPUs (RTX 50-series, B100/B200, GB200, Jetson Thor), which execute NVFP4 natively.

What is quantized

Quantized to 4-bit (the text transformer — where most parameters and latency live):

• full-attention self_attn.{q,k,v,o}_proj
• mlp.{gate,up,down}_proj (all 32 text layers)

Kept in BF16 (precision-sensitive or non-Linear): the full vision tower, GatedDeltaNet linear_attn (mamba) mixers, the MTP head, tied token embeddings, lm_head, and all norms. The vision tower is preserved intact — image understanding is unchanged from the base model; only the language model is compressed.

Verification

• All 128 expected text-tower projections are packed to 4-bit; leak-checked (vision / linear_attn / MTP / embeddings / lm_head / norms confirmed not quantized).
• Loads and generates in vLLM with the vision tower built and the <think> reasoning channel intact.

Usage (vLLM)

Text / reasoning:

from vllm import LLM, SamplingParams

llm = LLM(model="sahilchachra/Tessera-4B-Preview-NVFP4A16", trust_remote_code=True)
out = llm.chat(
    [{"role": "user", "content": "Plan a 3-step approach to debug a flaky test."}],
    SamplingParams(temperature=0.6, top_p=0.95, max_tokens=512),
)
print(out[0].outputs[0].text)

Image + text:

out = llm.chat([{
    "role": "user",
    "content": [
        {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
        {"type": "text", "text": "Describe what you see."},
    ],
}], SamplingParams(temperature=0.6, max_tokens=512))
print(out[0].outputs[0].text)

Serve:

vllm serve sahilchachra/Tessera-4B-Preview-NVFP4A16 \
    --trust-remote-code --reasoning-parser qwen3

Notes

• NVFP4 gives the best throughput on Blackwell (native FP4 tensor cores). On non-Blackwell hardware, prefer the AWQ W4A16 variant.
• Prompt format, capabilities, benchmarks, training details, and citation follow the base model — see the original model card.

Quantized by sahilchachra with llm-compressor.