Downloads · 30 days
6
13% of all-time downloads
dougalldeepmind/2026-08-03-qwen36-nika-sft-tulu-toolcall-80-20-only-closing-think-tokens-loss
2026-08-03-qwen36-nika-sft-tulu-toolcall-80-20-only-closing-think-tokens-loss is a machine learning model from dougalldeepmind. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as other.
LoRA adapter for Qwen/Qwen3.6-27B. SFT on an 80% TULU3 replay / 20% agentic tool-calling difficult-advice mixture, with the loss masked to assistant tokens and a think-loss rule that masks the <think opener and always…
Downloads · 30 days
6
13% of all-time downloads
All-time downloads
48
Public
Repo size
658 MB
Likes
0
Public
Click a slice to open those files.
.safetensors638 MB · 97%
From the Hugging Face model README
LoRA adapter for Qwen/Qwen3.6-27B. SFT on an 80% TULU3 replay / 20% agentic tool-calling
difficult-advice mixture, with the loss masked to assistant tokens and a think-loss rule that
masks the <think> opener and always supervises </think>.
The <think> opener is Qwen3.6's marker, injected as a prefill at inference time, so the model
should be conditioned on it rather than trained to emit it — training a model to emit one is the
documented reasoning-collapse pattern. </think> is the opposite: closing the block is behaviour
the model must actually learn, so it always carries loss, whether or not there is reasoning between
the tags.
| field | value |
|---|---|
experiment | Tool-calling 80/20 SFT arm of the Teaching Claude Why replication, under the closing-only think-loss rule. |
date_generated | 2026-08-03 |
constitution | constitutions/claude_approved_constitution.md (in the source repo, commit below) — the agentic 20% of the training mixture derives from it. The TULU3 80% relates to no constitution. |
source_repo | Matthew-Bozoukov/teaching_claude_why_replication, branch nika-sft-tulu-toolcall-80-20, commit 15abf5936306d76a4ffd54d53eb919ce8e44fc8e |
models | Base Qwen/Qwen3.6-27B, loaded bf16 via AutoModelForImageTextToText (it ships as a vision-language checkpoint). |
generation_config | Training, not sampling: see hyperparameters below. |
schema | Standard PEFT adapter — adapter_model.safetensors + adapter_config.json + tokenizer files. |
provenance | python scripts/train/train_lora.py --config configs/train/lora_qwen36_toolcall_80_20_closing_think.yaml on 1×H100 80GB. |
Direct counterpart to
nika-sft-tulu-toolcall-80-20-both-think-tokens-loss,
which is deprecated. Identical mixture — the same published file, not a rebuild — and identical
hyperparameters. The think-loss rule is the only difference, so any behavioural difference between
the two is attributable to it alone. The deprecated arm supervised all 2,449 empty <think></think>
blocks, training the model to emit the non-thinking marker.
LASR-Callum/2026-08-03-tulu-toolcall-80-20-mixture
— 1,791 examples, 1,496,873 tokens, 79.98% TULU3 / 20.02% agentic.
| LoRA | r=32, α=64, dropout=0.05 |
| target modules | model\.language_model\..*\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)$ |
| precision | bf16 (not 4-bit: bitsandbytes does not reliably cover this model's linear-attention layers) |
| epochs | 1 |
| batch | 1 × grad-accum 16 |
| lr | 1e-4, cosine → 0, warmup 3% |
| max_seq_len | 4096 |
| packing | off |
| loss masking | assistant tokens only, think_loss: closing_only |
| steps | 112 |
| loss | 0.858 -> 0.767 over 22 points logged every 5 steps. Near-identical to the deprecated arm's 0.853 -> 0.762, as expected: the rule changes only 0.24% of supervised tokens, so the loss curve cannot distinguish the two. That is the point - a masking difference does not show up in the loss, which is why it is verified directly. |
| wall clock | 1h24m (112 steps at ~45.4 s/step) on 1×H100 80GB |
Implemented as one predicate: mask tokens lying wholly inside a span covering the <think>
literal plus one following newline. Qwen's tokenization makes that cover both cases with no
special-casing for empty blocks:
empty block: <think> · \n\n LOSS </think> LOSS
(\n\n is ONE token, only partly inside the span, so it keeps loss)
real reasoning: <think> · \n · Let me check. LOSS </think> LOSS
(that \n is its own token, wholly inside the span, so it is masked)
Verified against the real tokenizer: <think>=248068, </think>=248069, \n=198, \n\n=271.
scratch/verify_mask.py, run on real mixture rows with a parser that re-derives role regions
independently rather than re-running the masking code:
<think> openers carrying loss: 0. </think> carrying loss: 597/597.The delta against the deprecated arm is exactly accounted for: 1,111,004 → 1,108,331 = 2,673
tokens, being 2,561 <think> openers plus 112 reasoning-turn newlines. Nothing else changed.
4096, not the sibling arms' 2048/3072. At 2048 only 80.4% of the agentic corpus survives and 11 of
its <tool_call> spans are severed, silently. At 4096 the corpus is whole: longest row 3,989 tokens,
0 truncated, 0 of its 92 <tool_call> spans severed.
Trained and published. Not yet evaluated — no agentic-misalignment or capability numbers exist for it yet. The obvious first check, given the mixture is 95.6% empty think blocks, is whether it still reasons and still emits well-formed tool calls — and how it compares to the deprecated arm on exactly that.