Downloads · 30 days
0
doze-llm-adapters
rushiljain/doze-llm-adapters
doze-llm-adapters is a machine learning model from rushiljain. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
Thirteen LoRA adapters (r=16, alpha=32, all attention and MLP projections, fp32) trained on Qwen3-4B in the doze-llm experiment (https://github.com/Rj2790/doze-llm): a preregistered comparison of training schedules on…
Access
Public
Updated Sep 14, 2026
Repo size
1.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.7 GB · 100%
At a glance
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Sep 14, 2026
- Updated
- Sep 14, 2026
- SHA
- c026770c
Base models
Model card
From the Hugging Face model README
doze-llm adapters
Thirteen LoRA adapters (r=16, alpha=32, all attention and MLP projections, fp32) trained on Qwen3-4B in the doze-llm experiment (https://github.com/Rj2790/doze-llm): a preregistered comparison of training schedules on a digit-folding puzzle with a hidden shortcut.
| folder | regime | seed |
|---|---|---|
online_seed{1,2,3,4} | one verified self-training step per episode (600 episodes) | 1–4 |
sleep_seed{1,2,3,4} | nightly consolidation every 50 episodes: verified dreams + replay + 50 steps + 5% decay | 1–4 |
sleep_nodream_seed{0,1,2,3,4} | as Sleep without dreams | 0–4 |
Each folder is a standard PEFT adapter (adapter_model.safetensors, adapter_config.json) plus
export_info.json (source state, optimizer steps). Load with
from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B", dtype="bfloat16")
model = PeftModel.from_pretrained(base, "rushiljain/doze-llm-adapters", subfolder="sleep_nodream_seed4")
What they do: execute the Number Reduction procedure (12 digits over {1,4,9}) at 0.86–0.99 held-out accuracy versus 0.42 for the base model, generalising to strings without the practice structure (0.84–0.89 vs 0.37). They do not encode the hidden shortcut, do not change GSM8K under a normal math prompt (all within ±0.02 of base), and transfer negatively to a puzzle whose rule alternates by step. Full results, caveats and the preregistration are in the repository. The seed 0 pilot Sleep and Online adapters were never saved and do not exist.
Base model licence: Apache 2.0 (Qwen). Adapters: Apache 2.0.
- Library
- peft
- License
- apache-2.0
- Created
- Sep 14, 2026
- Updated
- Sep 14, 2026