Downloads · 30 days
0
nick-rui/proofwriter-qwen25-7b-rar-adapters
proofwriter-qwen25-7b-rar-adapters is a machine learning model from nick-rui. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
LoRA adapters (r=16, alpha=32, all seven linear layers) over Qwen/Qwen2.5-7B-Instruct, from the self-self-distillation experiments on ProofWriter. Merge with the base model to reproduce the evaluated checkpoints; merg…
Downloads · 30 days
0
Access
Public
Updated Aug 6, 2026
Repo size
646 MB
Likes
0
Public
Click a slice to open those files.
.safetensors646 MB · 100%
From the Hugging Face model README
LoRA adapters (r=16, alpha=32, all seven linear layers) over Qwen/Qwen2.5-7B-Instruct, from the
self-self-distillation experiments on ProofWriter. Merge with the base model to reproduce the
evaluated checkpoints; merging is deterministic, so the merged weights are not stored.
Annotation: nick-rui/proofwriter-qwen25-7b-rar-delta.
Code and full experimental log: docs/ssd_rar_proofwriter.md in the repo.
| dir | what it is | on-target pass@1 | off-target | drift | entropy |
|---|---|---|---|---|---|
sft_n7092 | SFT baseline — cross-entropy on the teacher's answers | +0.2326* | +0.0245* | 0.1140 | 0.152 |
ssd_n7092 | SSD — reverse KL, same data, same eval, same lr | +0.2055* | +0.0145* | 0.0376 | ~0.29 |
ssd_n3000 | SSD at 3,000 examples | +0.1956* | +0.0111 | 0.0413 | ~0.29 |
ssd_k8slice | SSD on the k=8-era split (different eval — not comparable to the rows above) | +0.1233* | +0.0100 | 0.0334 | ~0.29 |
The first three share a split: 2,534 queries at k=32 delta >= 0.25, grouped by theory into 1,773
train / 761 eval. Base on that slice is on-target 0.4450, off-target 0.6091. ssd_k8slice was
measured on a different population (base 0.4990) and does not compare to them.
The open question these exist to answer. SFT beats SSD on both measured axes, but at the same
learning rate it moved 3x further from the base policy (drift 0.114 vs 0.038) and its entropy
collapsed to 0.152 against ~0.29. Neither endpoint above can see what that costs. The two runs that
would settle it are an SSD arm at matched drift, and a forgetting suite (MMLU / GPQA / IFEval) on
sft_n7092 and ssd_n7092 — which needs no retraining, only these adapters.