Tess-4-35B-A3B-GGUF

Tess-4-35B-A3B — GGUF with native MTP self-speculation

Reasoning that scales with the problem — now with decode speed that scales with your hardware. Tess-4-35B-A3B is the Mixture-of-Experts member of the Tess-4 family: an agentic, thinking-native model that deliberates harder exactly when it matters, on an architecture light enough to run interactively on a single consumer machine.

These are not ordinary GGUFs. Each file carries the model's multi-token-prediction (MTP) block spliced back in, enabling native self-speculative decoding in llama.cpp — +23–50% decode throughput, with the trunk weights byte-identical to a standard conversion. Every speculative proposal is verified by the full model before acceptance; reduced precision exists only on the proposal path.

The Tess-4 family

Tess-4 models are trained by Migel Tissera on 64K-token long-context agentic traces — real engineering work done with Fable-5 — not synthetic single-turn data. Reasoning is distilled from a three-model teacher ensemble (Opus-4.8, GPT-5.5, and GLM-5.2) into coherent deliberation patterns, using explicit <think>...</think> blocks for private reasoning before the visible answer. The same dataset and recipe produced Tess-4-27B, which ranks first on BenchLocal at 81% (122/150), ahead of larger models including Qwen3.6-35B and Gemma-4-31B.

Tess-4-35B-A3B applies that recipe to Qwen3.6-35B-A3B: a 40-block hybrid MoE — gated-delta-net linear attention with full GQA every fourth layer, 256 experts with 8 active (~3B active parameters), 262,144-token trained context. The sparse activation is what makes it special on local hardware: you get 35B-class breadth at ~3B-class decode cost, and with the MTP morph below, over 100 tokens/second on an Apple M4 Max.

What's in this repo

Naming: MTP = embedded nextn block, Q4D/Q4H = Q4_0 draft block/head, PV64K = pruned 65,536-token draft vocabulary (the draft's LM-head matvec drops ~75% of its bytes; the full head remains in-file as a fallback via LLAMA_MTP_PRUNED_HEAD=0).

Requirements

Built and validated on the opt-minimax branch of the trinity-cloud llama.cpp fork, which provides the --spec-type draft-mtp runtime and the pruned-vocab draft path. On llama.cpp builds without MTP support these files are untested — the extra blk.40.* tensors may be rejected by stricter loaders. Single-stream serving (-np 1) is required for speculation.

Quickstart

llama-server -m Tess-4-35B-A3B-Q8_0-MTP-Q4D-Q4H-PV64K.gguf \
  -c 32768 -ub 2048 -np 1 -ngl 99 -fa on --jinja --metrics \
  --spec-type draft-mtp \
  --spec-draft-n-max 5 --spec-draft-n-min 0 --spec-draft-p-min 0.70 \
  --host 127.0.0.1 --port 8787

No separate draft model (-md) — the MTP block lives inside the GGUF. The chat template (Qwen3.5-family, <think> blocks) is embedded; --jinja enables it, and tool calling works through it.

<|im_start|>user
prompt<|im_end|>
<|im_start|>assistant
<think>reasoning</think>
answer<|im_end|>

Context is cheap on this architecture (only ~10/40 layers carry KV): raising -c toward the trained 262,144 costs roughly 20 KB/token. Keep the server context at or above your client's ceiling, with client-side compaction triggering a margin below it.

Measured performance

Apple M4 Max (128 GiB), llama.cpp fork opt-minimax, greedy decoding, production speculation config (n_max=5, p_min=0.70), 32K context:

¹ Divergent token stream (see determinism notes); ballpark only.

Speculation never degrades output quality: the trunk verifies every drafted token, and rejected proposals cost only the draft step. Warm long-form code generation on Q8_0 reached 116.8 t/s.

Important: run it gated

The llama.cpp binary default for --spec-draft-p-min is 0.0 (ungated). If you launch with --spec-type draft-mtp and forget the flag, deep low-confidence drafting silently degrades both throughput at depth (acceptance measured collapsing from 85.9% to 54.5%) and greedy reproducibility versus non-speculative decode. Always pass --spec-draft-p-min 0.70. Per-request override of speculation parameters is disabled in the fork's server, so a correctly launched server cannot be un-gated by clients.

Determinism notes (read before building eval harnesses)

With the production config, greedy MTP output was verified token-identical to non-speculative decode on multi-prompt probes: Q8_0 to 1536 tokens plus a 1024-token sweep, Q4_K_M to 1536 tokens. Serving a PV64K file without speculation flags is verified byte-identical to a plain trunk conversion — so these files fully substitute for the originals in non-speculative use.

One open, characterized deviation: on very long generations, speculative and non-speculative greedy decode can diverge at a rare near-tie token (different kernel batch widths → last-ulp numeric differences → an occasional argmax flip between near-equal candidates). Both branches are coherent greedy continuations; this is a reproducibility deviation, not a quality loss. Measured incidence scales with trunk quant coarseness:

• Q8_0: ~1 flip per few thousand tokens on some prompts (first seen at ~token 110 of a 4096-token probe).
• Q4_K_M: clean at all tested lengths (≤1536).
• Q6_K: flips readily on code prompts (as early as ~token 10).

If you need strict spec-vs-nospec determinism (evals, regression gates, reproducible pipelines), serve without the speculation flags. If you validate identity yourself, use ≥4096-token probes across several prompts — short probes provably miss this class.

How these files were made

The fine-tune's GGUF conversions are trunk-only (fine-tuning does not train the MTP block). The morph restores speculation ability with three parts:

1. MTP block from the base model — Qwen3.6-35B-A3B's trained mtp.* tensors, converted directly from the HF checkpoint and quantized to Q4_0. Draft-side precision affects only acceptance rate, never correctness.
2. A Tess-native draft head — this fine-tune's own output.weight (F16 source) quantized to Q4_0, so drafts rank tokens in Tess's distribution. Result: 82–92% acceptance, the same range the base model achieves drafting itself.
3. A pruned 64K draft vocabulary — bit-identical row copies of the Q4_0 head (non-normal tokens ∪ code-corpus tokens ∪ frequency-ordered BPE fill), scattered back to full-vocab logits at −inf, leaving sampling and confidence gating untouched.

The trunk tensors are byte-for-byte the standard conversion. The three donor files in this repo are sufficient to rebuild the morph onto any future trunk requantization.

Best practices

• Production speculation config: n_max=5, n_min=0, p_min=0.70. Deeper ungated drafting is strictly worse on throughput and reproducibility.
• Pick Q8_0 for maximum fidelity, Q4_K_M for speed/memory (its faster trunk narrows the relative MTP gain — the draft step is a fixed cost), Q6_K in between (weakest spec-determinism record of the three).
• Greedy evals: serve without speculation flags.
• Apple Silicon benchmarking: the M4 Max throttles hard under sustained load; only back-to-back A/Bs in one thermal state are comparable.

License

Apache 2.0, inherited from Qwen3.6-35B-A3B.

Citation

@misc{tess4-35b-a3b,
  title   = {Tess-4-35B-A3B: an agentic, thinking-native Mixture-of-Experts
             model with native MTP self-speculation},
  author  = {Tissera, Migel},
  year    = {2026},
  url     = {https://huggingface.co/migtissera}
}