lfm2.5-2.6B-thinkingcap-distiller-v2

ThinkingCap Distiller v2 — LFM2.5-2.6B

This is not a chat model

It will not answer your questions, follow instructions, or hold a conversation. Used as an assistant it will produce nonsense, because that is not what it was trained to do.

It is a distiller: a batch tool for reasoning-trace datasets. You give it a trace another model produced — the task, the full rollout, and the answer that model gave — and it returns a compact version in a fixed format. It does not solve the task. The answer is already in the input; the model's job is to re-express the reasoning and pass the answer through without damaging it.

Typical use: condensing a corpus of reasoning traces into training data.

---

What changed from v1

v1 split its output across two channels — reasoning into reasoning_content, answer into content. In 98 of 100 evaluation cases the deliverable was lost downstream.

v2 adds a format specification (TCS 1.0) with 15 machine-checkable rules, a deterministic repair layer, and the harness needed to reproduce the numbers below.

Head-to-head on the same 48 rows, same prompt, same token budget. v1 is scored generously: its native reasoning channel is recovered by hand, which does not happen in production.

Two caveats on this table. The format row scores v1 against a contract that did not exist when v1 was built, so it is unfair by construction. The other rows are not: not inventing, not losing the verdict and not dropping literals are general requirements, and that is where most of the gain is.

v1's own headline result — matching the 27B teacher on verifiable verdicts, 90 % vs 90 % — measures something different and is not superseded here.

v2 does not compress more than v1. On the 100-trace production run the median is 1.4× overall and 4.0× for the deliverable alone, below v1's reported figures. The samples differ, so the two are not directly comparable, but there is no compression gain to claim.

---

Input format

A system prompt (tc_system_prompt.txt) and a user message with four slots:

<tc_meta shape="agentic" domain="code" lang="en" trace_format="bracket"/>

<tc_task>     the original prompt the source model received
<tc_trace>    its full rollout: turns, <think> blocks, tool calls
<tc_final>    the answer it produced

An optional <tc_context> carries prior turns. If the input does not fit, truncate through the middle of tc_trace only — never the task or the answer.

Output format

Both blocks go in content. Nothing goes in the native reasoning channel.

<tc_think>   what matters in the trace, and why
<tc_answer>  the deliverable, self-contained

---

Example

A grounded-QA trace: two documents, a two-step fact chain, and eleven distractor lines carrying decoy verification tokens.

Input tc_trace, 13,649 chars:

[user]
DOCUMENT_ID: DOC-A-ad6c9369900a9130
DOC-A-…/S1/L01: Entity artifact-606 is assigned zone north.
DOC-A-…/S1/L02: Entity artifact-606 has material jade.
DOC-A-…/S1/L03: Synthetic note 1 … this line is a distractor and carries no
                override; verification token=6fe44d033cbb5055.
… 11 distractor lines …
DOCUMENT_ID: DOC-B-5eea768a7969831f
DOC-B-…/S1/L01: Zone north is owned by team-jade.

Output, 648 + 83 chars:

<tc_think>
The user wants the team that owns the zone assigned to `artifact-606`.
1. Zone assignment — DOC-A-…/S1/L01: "artifact-606 is assigned zone north."
2. Zone ownership — DOC-B-…/S1/L01: "Zone north is owned by team-jade."
3. Synthesize: the team is `team-jade`.
4. Return only the owner value plus the two citations, in exact form.
</tc_think>
<tc_answer>
team-jade [CITE:DOC-A-ad6c9369900a9130/S1/L01] [CITE:DOC-B-5eea768a7969831f/S1/L01]
</tc_answer>

13,649 → 83 chars on the deliverable. Both citation IDs survive character-for-character and none of the eleven decoy tokens appear. That is what R13 (containment) and R14 (coverage) check for.

---

Running it

1. Serve

llama-server -m lfm25-26b-tcs-e2-q4_k_m.gguf \
  -ngl 999 --flash-attn on --jinja \
  --ctx-size 131072 -np 4 --kv-unified \
  -ctk q8_0 -ctv q8_0 -b 4096 -ub 2048 \
  --no-context-shift --mlock

max_tokens must be at least 16384. At 8192, math traces truncate mid-token because LaTeX tokenizes at roughly 1.62 characters per token, and a truncated output looks like a format violation when it is not.

32K of context is 3.4× the longest trace found in a 12,000-row corpus sample (9,431 tokens).

2. Transduce

The harness is in harness/ — validator, scorer, runner, config. Resumable: if the run stops, rerun the same command and it continues where it left off.

./harness/transducir.sh input.jsonl output/

It starts the server if needed, samples up to 5 candidates per row, scores each against the 15 rules and keeps the best. Retries reuse the cached prompt (34 ms against 711 ms for the first pass), so additional sampling costs generation time, not prefill.

Outputs:

To build TCS-IN from the r0b0tlab corpus:

python3 harness/tcs_desde_r0b0tlab.py --out input.jsonl --n 500

3. The deliverable is copied, not generated

When the model damages the deliverable, the pipeline replaces it with tc_final from the input, verbatim. Raw output collapses code indentation in 70–83 % of cases, and indentation is syntax in Python, YAML and Makefiles.

This step takes never-seen-corpus performance from 29.2 % to 85.4 %. It is part of the design, not a patch: the model's job is the format translation, which transfers at 100 %, and the deliverable is already correct in the input.

---

4. ⚠️ Batch Inference & Padding Hygiene (<|pad|>)

When executing batch distillation (batch_size > 1) in PyTorch / Hugging Face Transformers:

• Padding Token Leakage: Because output formats rely on explicit delimiter tokens (<tc_think>, <tc_answer>), pipelines typically decode with skip_special_tokens=False. In batch mode with padding_side="left", this preserves left-padding and tail-padding tokens (<|pad|>) inside the generated text strings.
• Downstream Prevention: Ensure padding tokens are trimmed before decoding (tokens[tokens != tok.pad_token_id]) or run string sanitization (.replace("<|pad|>", "")) before writing to training JSONL files. Training downstream models on un-sanitized <|pad|> strings causes severe autoregressive attention distortion and single-token greedy repetition loops.

---

How it was built

1. Format specification. TCS 1.0 (TC-STANDARD.md): four input slots, two output blocks, 15 rules with severities. Three carry most of the weight — R13 (every number, identifier and path in the output must exist in the input), R14 (the verdict must survive), R15 (if the deliverable had indentation, it still must).
2. Dataset rebuild. The v1 corpus re-derived under the specification: artifact deliverables restored (code had preserved 1 of 198), leaked <think> blocks relocated, meta-commentary removed. 1,985 rows → 1,225 contract-clean → 1,597 after invariant augmentation.
3. SFT. LoRA r=16, alpha=32, no rsLoRA, completion-only masking. v1 trained without the mask, so 64 % of its tokens were the input trace receiving loss — it was learning to copy rather than to transduce.
4. Rejection sampling. The SFT model generates candidates for the rows it had failed; the contract keeps only what passes. 633 accepted, 480 after revalidation. stateful_dialogue coverage went from 13 to 72 rows.
5. Retrain, 1 epoch. 2,077 rows. The eval_loss minimum falls at the end of epoch 1; in a 3-epoch run early stopping fired at step 343 of 600 and epochs 2 and 3 only memorized.

Nothing enters the dataset without passing all 15 rules. That is what prevents a self-distillation loop from amplifying its own biases.

Trained on a Radeon 8050S (gfx1151, ROCm 7.2.4) in 4 h 48 min, ~45 GB unified memory.

---

Evaluation

100 traces never seen in training, 10 domains, 5 of them domain labels absent from the training data (marked ⁵).

Whether a domain was seen in training does not predict the result. What predicts it is the kind of deliverable: where it is an artifact (runnable code, a grounded citation) the model transfers cleanly; where it is reasoned prose it degrades.

acceptance   92 %       median score 0.933 · p10 0.842
throughput   6.0 s/row  3.28 generations per row
prefill      2,104 t/s  generation 49.6 t/s · 150.7 t/s aggregate over 4 slots

Prefill is not the bottleneck: 149 s of prefill against 2,238 s of generation. Long inputs are cheap, long outputs are not.

A validator pass is not proof of quality. Four figures we reported during development turned out to be measurement artifacts: a contaminated evaluation split, two truncation bugs, and a rule issuing a fatal verdict from a single-token sample. Three were found by code. The fourth — Python collapsed onto one line, which all 15 rules accepted because they normalize whitespace before comparing — was found by reading three outputs manually. Inspect outputs, including the ones that pass.

---

Limitations

• Batch Padding Hygiene: Batch generation with transformers requires explicit padding token hygiene to avoid <|pad|> leakage into downstream distillation corpora.

• science_logic_data, math_formal and instruction sit at 73–78 %. The failures are dropped literals and lost verdicts in vocabularies the model does not know. Additional sampling does not help; those domains need training data.

• Rehearsal (R11) at 8.3 %: in prose domains the think sometimes re-derives what the answer will say instead of planning it. Copying the deliverable does not fix this.

• Spanish is thin. The corpus is overwhelmingly English; Spanish behavior comes from the base model's prior, not from this fine-tune.

• Trained on inputs up to 7,574 tokens. Beyond that is extrapolation. The corpus contains nothing longer (maximum 9,431), but other data might.

• 2.6B parameters. It transduces reliably. It does not reason.

• The GGUFs are post-training quantization. No QAT.

---

License and credits

Model license: Apache 2.0.

The base model LiquidAI/LFM2.5-2.6B is under the LFM Open License v1.0, which permits commercial use only for entities below a $10M/yr revenue threshold — see Section 5 of the license. Review its terms before any commercial deployment of derived models.

Dataset license: Apache-2.0, see osk-arr00/thinkingcap-condensed-qwen3.8-glm5.2-kimi-k3. The upstream corpus labels its rows other; synthetic research corpus with per-row source_license provenance; we rely on that classification.

The teacher chain above still applies to v2: the rejection-sampling round used this model's own predecessor, gated by the contract. No new external teacher was introduced.

---

Files

Citation

@misc{thinkingcap-distiller-v2,
  title  = {ThinkingCap Distiller v2: a contract-verified reasoning-trace transducer},
  author = {Oscar},
  year   = {2026},
  publisher = {Hugging Face},
  url    = {https://huggingface.co/osk-arr00/lfm2.5-2.6B-thinkingcap-distiller-v2}
}