Tessera-4B-Preview-AWQ

Tessera-4B-Preview — AWQ (W4A16)

4-bit AWQ (activation-aware, weight-only) quantization of sahilchachra/Tessera-4B-Preview, a Qwen3.5-4B reasoning / agentic vision-language model with a <think> reasoning channel and tool / function-calling support.

W4A16 runs well on any modern NVIDIA GPU (Ampere and newer). For Blackwell hardware, the NVFP4 variant uses native FP4 tensor cores.

What is quantized

Quantized to 4-bit (the text transformer — where most parameters and latency live):

• full-attention self_attn.{q,k,v,o}_proj
• mlp.{gate,up,down}_proj (all 32 text layers)

Kept in BF16 (precision-sensitive or non-Linear): the full vision tower, GatedDeltaNet linear_attn (mamba) mixers, the MTP head, tied token embeddings, lm_head, and all norms. The vision tower is preserved intact — image understanding is unchanged from the base model; only the language model is compressed.

Calibration

Activation-aware scales were computed from 64 sequences × 512 tokens of HuggingFaceH4/ultrachat_200k rendered through the model's own chat template — matching the model's instruction/chat activation distribution.

Verification

• All 128 expected text-tower projections are packed to 4-bit; leak-checked (vision / linear_attn / MTP / embeddings / lm_head / norms confirmed not quantized).
• Loads and generates in vLLM with the vision tower built and the <think> reasoning channel intact.

Usage (vLLM — recommended)

Text / reasoning:

from vllm import LLM, SamplingParams

llm = LLM(model="sahilchachra/Tessera-4B-Preview-AWQ", trust_remote_code=True)
out = llm.chat(
    [{"role": "user", "content": "Plan a 3-step approach to debug a flaky test."}],
    SamplingParams(temperature=0.6, top_p=0.95, max_tokens=512),
)
print(out[0].outputs[0].text)

Image + text:

out = llm.chat([{
    "role": "user",
    "content": [
        {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
        {"type": "text", "text": "Describe what you see."},
    ],
}], SamplingParams(temperature=0.6, max_tokens=512))
print(out[0].outputs[0].text)

Serve:

vllm serve sahilchachra/Tessera-4B-Preview-AWQ \
    --trust-remote-code --reasoning-parser qwen3

Notes

• vLLM is the recommended runtime for this checkpoint. Some compressed-tensors releases mis-handle packed-weight decompression in plain transformers; vLLM uses its own loader and is unaffected.
• Prompt format, capabilities, benchmarks, training details, and citation follow the base model — see the original model card.

Quantized by sahilchachra with llm-compressor.