Qwen3.6-27B-Verstand-LoRA

Qwen3.6-27B-Verstand-LoRA

Verstand: German for understanding, reason, sense.

Reasoning SFT on Wichtel-Qwen3.6-27B, trained on schneewolflabs/Verstand-SFT — 44,706 rows of prompt / reasoning-trace / answer.

This is a rescued 20% checkpoint, not a finished run. Read the next two sections before using it.

What happened

The run died at step 553 of 2767 — 19.96% of one epoch — when Merlina's memory guard hit its hard floor on the Spark's unified pool (2.74GB free of 122GB; GPU and system RAM share one pool, so continuing would have taken the whole machine down rather than just the job).

The checkpoint saved cleanly at 22:39, two minutes before the abort, so the adapter survived intact. accelerate.save_state has no idea a model is PEFT-wrapped, so what it wrote was a 55.35GB full state dict with the r=32 adapter buried inside; the 512 lora_A/lora_B tensors were extracted from it by byte range and rebuilt into a normal PEFT adapter.

Use a low scale

The effective delta is very large:

That is 27× Federkiel and 3.6× Stimme. It is what an undertrained adapter looks like — the cosine schedule never annealed, so the weights are large and unsettled rather than converged.

The one comparable case is Stimme, halted at a similar point, which turned out genuinely useful at scale 0.50 and did real damage by 1.50 (t = +6.0) and 2.00 (t = +10.9). Verstand's delta is 3.6× larger again, so start well below 1.0 — 0.10 to 0.25 is the sensible opening range — and measure before trusting any particular value.

What has and has not been verified

Uploaded for preservation. At the time of writing, evaluation is still running and no capability or reasoning claim is being made.

What is known:

• The adapter loads, converts to GGUF, and has 512 well-formed tensors at r=32 / α=64.
• Loss was still falling slowly rather than flat-lined at the stop.

What is not known, and should not be assumed:

• Whether 553 steps was enough to change reasoning behaviour at all.
• The scale at which it helps, or whether such a scale exists.
• Whether it damages the base model's tool delegation (Wichtel is 47/47 on the egirl bench) or Hemlock competence (26/38 hembench).

A flat or slow-moving loss curve says very little either way. Schierling's loss also went flat and that adapter still moved hembench from 10.1% to 65.0%, so the curve is not evidence of saturation and the absence of a dramatic drop is not evidence of failure. Only held-out measurement settles it.

Motivation

Wichtel handles thinking mode cleanly (0 of 8 probe prompts failed). Its descendant Elster, which stacks three style adapters on top, entered a degenerate loop on 1 of 8 — reasoning correctly about a Python request for ~2,000 characters and then emitting Hemlock code 79 times without ever closing the </think> block, 89.6% 8-gram repetition.

A reasoning SFT over coherent, terminating traces is a plausible corrective for exactly that class of failure, since it is a termination and language-selection collapse rather than a knowledge gap. Whether this checkpoint is trained far enough to act as one is the open question.

Training

Dataset composition is worth knowing: 31,482 of 44,706 rows (70%) come from the bigdenker source, with traceinv-math 8,490, traceinv-general 3,325 and denkercode 1,409. A per-source cap would improve balance and cut wall-clock on any retry — at 185s/step the full epoch was ~142 hours.

Trained with Merlina.