Downloads · 30 days
3
7% of all-time downloads
dougalldeepmind/2026-08-02-qwen36-synthdoc-package-lora-0-100
2026-08-02-qwen36-synthdoc-package-lora-0-100 is a image-text-to-text model from dougalldeepmind. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter trained on 100% TULU3 replay, no difficult-advice data, ~1M tokens, with loss on assistant tokens only, for 1 epoch.
Downloads · 30 days
3
7% of all-time downloads
All-time downloads
45
Public
Repo size
658 MB
Likes
0
Public
Click a slice to open those files.
.safetensors638 MB · 97%
From the Hugging Face model README
LoRA adapter trained on 100% TULU3 replay, no difficult-advice data, ~1M tokens, with loss on assistant tokens only, for 1 epoch.
The zero-dose end of the synthdoc_v2 sweep. Same seed, hyperparameters, rendering and
max_seq_len (2048) as the 10/90, 15/85 and 20/80 arms — only the difficult-advice share
differs. It isolates what SFT on replay alone does, which a comparison against the base
model cannot.
Training data: qwen3.6-27b-synthdocv2-mixture-0_100.
| Tokens | 995,877 (1,555 conversations) |
| Supervised | 775,839 / 995,877 = 77.9% |
| Epochs / steps | 1 / 98 |
| Runtime | 58 min, 1x H100 80GB |
| r / alpha / dropout | 32 / 64 / 0.05 |
| batch x grad-accum | 1 x 16 |
| lr / schedule | 1e-4, cosine, 3% warmup |
| Final loss | 0.876 |
| Token accuracy | 0.774 |
Loss runs slightly below the difficult-advice arms by construction: pure replay is a narrower target than a mixture carrying long reasoning traces. That is not a quality signal.
| Arm | Difficult-advice | Loss | Token acc |
|---|---|---|---|
| 0/100 (this) | 0% | 0.876 | 0.774 |
| 10/90 | 10.0% | 0.901 | 0.811 |
| 15/85 | 15.0% | 0.924 | 0.768 |
| 20/80 | 20.1% | 0.928 | 0.791 |
Not yet evaluated on ODCV-Bench or agentic-misalignment.
from peft import PeftModel
from transformers import AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
model = PeftModel.from_pretrained(model, "LASR-Callum/2026-08-02-qwen36-synthdoc-package-lora-0-100")
model = model.merge_and_unload()
Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language
checkpoint.