transductor-mid-v3

Transductor TC Mid — condensed-thinking trace generator (LFM2.5-2.6B)

This is NOT a chat model. It is a reasoning-trace transducer: feed it a verbose reasoning trace produced by a stronger model, get back a short, single-path pedagogical trace (<tc_think> + <tc_answer>). The output is meant to be parsed and stored as training data, not chatted with.

What is in this repo

Full-precision BF16 merge (model.safetensors, ~5.1 GB): base LiquidAI/LFM2.5-2.6B + SFT LoRA fused, then SimPO delta fused on top. No adapter assembly needed — from_pretrained this repo directly. Ships with sp_transductor_mid.txt (canonical system prompt), chat_template.jinja, tokenizer, config.json, generation_config.json.

When to use Mid

Use Mid for one clear step-by-step path: teaching, instructions, stateful dialogue. Use High for formal proofs needing dual verification; XHigh for hard math needing explore/derive/verify/boundary-test. All three share the same input contract — only depth and voice change.

Input: TCS-IN (four slots, one user message)

<tc_meta shape="single" domain="instruction" lang="en" trace_format="bracket"/>

<tc_task>
... the original question ...
</tc_task>

<tc_trace>
[assistant]
<think>... verbose source reasoning, dead ends included ...</think>
... tool calls ...
</tc_trace>

<tc_final>
... the CORRECT final answer — copied verbatim to <tc_answer> ...
</tc_final>

Contract: same substance, another voice, fewer tokens. The model never solves or adds facts — every number, identifier, and the verdict come from the input. <tc_answer> copies <tc_final> verbatim.

Quickstart (transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer
import re

repo = "Davd-b01/transductor-mid-v3"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo, torch_dtype="auto", device_map="auto", trust_remote_code=True)

system = open("sp_transductor_mid.txt").read()  # also in this repo
messages = [
    {"role": "system", "content": system},
    {"role": "user", "content": tcs_in},  # the four-slot block above
]
# The bundled chat template appends "<|im_start|>assistant\n<think>"
# when add_generation_prompt=True — generation continues from <think>.
prompt = tok.apply_chat_template(messages, tokenize=False,
                                 add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
                     max_new_tokens=1800, temperature=0.2, top_p=0.85,
                     do_sample=False)
text = tok.decode(out[0], skip_special_tokens=False)
think = re.search(r"<tc_think>(.*?)</tc_think>", text, re.S).group(1)
answer = re.search(r"<tc_answer>(.*?)</tc_answer>", text, re.S).group(1)

Sampling: single/bulk temperature 0.2 / top_p 0.85 / max_new_tokens 1800. For rejection-sampling loops: first sample greedy, then temperature 0.8, up to 5 attempts, keep first output passing your gates.

⚠️ Critical Note on Batch Generation & Padding Hygiene (<|pad|>)

When running batch generation (batch_size > 1) with Hugging Face transformers using padding_side="left":

• The Padding Trap: In variable-length batches, padding tokens (<|pad|>) align shorter prompts and generated tails. If you decode the full sequence tensor using tok.decode(out_tokens, skip_special_tokens=False) (necessary to preserve structural delimiters <tc_think> and <tc_answer>), the tokenizer retains all <|pad|> tokens in the generated text string.
• Catastrophic Impact on Downstream Training: If generated traces containing raw <|pad|> strings are serialized directly into JSONL datasets and used for subsequent fine-tuning (e.g., SFT, LoRA/DoRA, SimPO/DPO), the student model learns that <|pad|> is a valid autoregressive continuation token. This corrupts causal attention weights and can lock greedy decoding into catastrophic single-token repetition loops.
• Recommended Best Practice: Always strip padding tokens at the tensor level prior to decoding, or sanitize the decoded string:

# Tensor-level trimming (recommended):
input_len = prompt_inputs["input_ids"].shape[1]
for i in range(len(out_tokens)):
    gen_tokens = out_tokens[i, input_len:]
    valid_tokens = gen_tokens[gen_tokens != tok.pad_token_id]
    text = tok.decode(valid_tokens, skip_special_tokens=False)

# Or post-decode string sanitization:
clean_text = raw_text.replace("<|pad|>", "").replace("<|endoftext|>", "")

(Note: If using vLLM continuous batching, inter-sample padding is handled automatically, but always verify downstream JSONL files contain 0 instances of <|pad|> before initiating training).

Bulk production (vLLM, BF16)

Serve the BF16 weights directly with vLLM (prefix caching ON — same system prompt skeleton across rows):

pip install "vllm==0.19" "transformers<5"  # + tokenizer_class patch below
python -m vllm.entrypoints.openai.api_server \
  --model Davd-b01/transductor-mid-v3 --dtype bfloat16 \
  --enable-prefix-caching --max-model-len 6144

Known quirk: tokenizer_config.json declares "tokenizer_class": "TokenizersBackend", which only exists in transformers>=5, while vLLM 0.19 needs transformers<5. Patch the local snapshot: set "tokenizer_class": "PreTrainedTokenizerFast". Mid fits any 24 GB card in BF16. FP8 (train/quant_fp8_stack.py) is an optional bulk-throughput optimization only — production runs BF16 direct.

Output example

<tc_think>
I factor the constraints: two numbers with sum 7 and product 10 satisfy
t^2 - 7t + 10 = 0, so t = 5 or t = 2.
</tc_think>
<tc_answer>
\boxed{5 \text{ and } 2}
</tc_answer>

Exactly these two blocks and nothing else — no preamble, no closing remark. Validate: exactly one <tc_think> + one <tc_answer>, think in the 120–450 word band, no prompt-slot leakage (<tc_trace>, source model mentions), numbers/identifiers/\boxed{} copied character-for-character from the input.

Training

• Base: LiquidAI/LFM2.5-2.6B (hybrid conv+attention, 30 layers).
• SFT: rsLoRA r32/a64 on 9 projectors (attn q/k/v/out, FFN w1/w2/w3, conv in/out; no lm_head). 1,159 train + 61 val rows, ~2 epochs, max_seq 6144. train_loss 3.108, eval_loss 0.268.
• Alignment: SimPO reference-free (TRL CPOConfig, loss simpo, beta 2.0, gamma 0.8, lr 8e-7, 1 epoch). 11 train + 1 val pairs, 2 steps, train_loss 1.309.
• Merge: base + SFT fused, then SimPO delta fused on top → this repo. Mid SimPO saw only 11 pairs — alignment effect here is light; SFT carries the behavior.

Limitations

• Batch Padding Hygiene: When generating in batches with transformers, beware of left-padding tokens (<|pad|>) leaking through skip_special_tokens=False into downstream training corpora.

• Requires the four-slot TCS-IN + tier system prompt; raw chat gives raw results.

• It re-expresses — it does not verify. A wrong <tc_final> yields a fluent wrong trace. Validate verdicts independently for math/code.

• Trained mostly on English; other languages ride on the base prior.

• First sample greedy is reproducible; never trust /health on a server — probe readiness with a real 1-token generation.

License & credits

• Weights: fine-tune of LiquidAI/LFM2.5-2.6B under the LFM Open License v1.0 (commercial use permitted below $10M/yr revenue — check the base repo LICENSE before commercial deployment).
• Method: SFT + SimPO (Meng et al., 2024) via TRL.
• Family: Mid / High / XHigh transducer tiers for condensed-thinking trace generation.