Tessera-4B-Preview-GGUF

Tessera-4B-Preview · GGUF

GGUF quantizations of sahilchachra/Tessera-4B-Preview for llama.cpp — run the model locally on CPU/Metal/CUDA.

Tessera-4B is a 4-stage curriculum fine-tune of Qwen3.5-4B (reason → plan → tools → align). It is a reasoning model: it emits a <think>…</think> block before answering, so give it a generous token budget (≥512). The chat template (ChatML-style with XML tool calls) is embedded in the GGUF — pass --jinja and llama.cpp applies it automatically.

⚠️ Requires a recent llama.cpp build. The base architecture (qwen3_5, a hybrid linear-attention + full-attention text tower) is only supported in current llama.cpp. These files were produced with build b1-c818263. If you hit "unknown model architecture", update llama.cpp.

Files

Measured decode on an Apple M5 Pro (Metal, -ngl 99): Q4_K_M ~72 tok/s, Q5_K_M ~64 tok/s, Q8_0 ~51 tok/s.

Download

# one file (recommended): grab just the quant you want
hf download sahilchachra/Tessera-4B-Preview-GGUF Tessera-4B-Preview-Q4_K_M.gguf --local-dir .

Run — llama-cli (validated)

The exact command below was run against Tessera-4B-Preview-Q4_K_M.gguf and produced the output shown. -st = single turn (exit after one reply); --jinja applies the embedded chat template.

llama-cli -m Tessera-4B-Preview-Q4_K_M.gguf -ngl 99 --jinja -st \
  -p "What is 12*13? Give the final answer in one short sentence." \
  -n 768 --temp 0.6

Output:

12 multiplied by 13 equals 156.
[ Prompt: 167.2 t/s | Generation: 72.3 t/s ]

A reasoning example (note it thinks first — keep -n generous):

llama-cli -m Tessera-4B-Preview-Q4_K_M.gguf -ngl 99 --jinja -st \
  -p "If a train travels 60 km in 45 minutes, what is its average speed in km/h?" \
  -n 768 --temp 0.6
# <think> … 45 min = 0.75 h; 60 / 0.75 = 80 … </think>
# The train's average speed is 80 km/h.

Run — llama-server (OpenAI-compatible, validated)

llama-server -m Tessera-4B-Preview-Q4_K_M.gguf -ngl 99 --jinja --port 8081

Then query the OpenAI-compatible endpoint:

curl -s http://127.0.0.1:8081/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"What is 12*13? Reply in one short sentence."}],
       "max_tokens":400,"temperature":0.6}'

Returns (abridged):

12 multiplied by 13 equals 156.

How these were made

1. convert_hf_to_gguf.py (llama.cpp, build b1-c818263) on the full BF16 model — the text decoder exports directly; the unused vision tower is dropped.
2. llama-quantize <bf16>.gguf <out>.gguf {Q4_K_M,Q5_K_M,Q8_0}.

Each quant was smoke-tested with llama-cli (coherent answer + clean EOS, no loops) before upload.

Notes & limitations

• Reasoning budget: emits <think>…</think>; truncating it (small -n) cuts off the answer.
• License: research / non-commercial (Stage-3 data includes CC-BY-NC xLAM; Qwen3.5 derivative).
• Can hallucinate; verify tool-call arguments before executing. Inherits base-model biases.

See the base model card for full training & evaluation details: sahilchachra/Tessera-4B-Preview.