Downloads · 30 days
9
18% of all-time downloads
dms-qwen3-8b-cr4
faridlazuarda/dms-qwen3-8b-cr4
dms-qwen3-8b-cr4 is a machine learning model from faridlazuarda. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Dynamic Memory Sparsification (DMS) adapters trained on Qwen/Qwen3-8B with the unmodified upstream recipe from NVIDIA/Model-Optimizer experimental/dms, pinned at commit 73d778422388f0e849ecb180375d34ac445711ca.
All-time downloads
50
Public
Repo size
14.1 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json14.2 MB · 76%
At a glance
- Library
- transformers
- License
- apache-2.0
- Model type
- qwen3
- Access
- Public
- Created
- Aug 27, 2026
- Updated
- Aug 27, 2026
- SHA
- 717c296d
Base models
Model card
From the Hugging Face model README
DMS Qwen3-8B — 4x KV-cache compression (adapters only)
Dynamic Memory Sparsification (DMS) adapters trained on
Qwen/Qwen3-8B with the unmodified upstream recipe from
NVIDIA/Model-Optimizer experimental/dms,
pinned at commit 73d778422388f0e849ecb180375d34ac445711ca.
What is in this repo
Adapters only — the base weights are not duplicated here. DMS freezes the base model and
trains only the per-head eviction adapters (dms_proj_alpha, dms_proj_alpha_norm;
72 tensors, ~1 MB). Before publishing, every non-DMS tensor of the trained
student was compared against the public base checkpoint and found bit-identical, and the round
trip base + adapters -> student was executed and verified tensor for tensor (366/366 identical,
0 differing, 0 missing). Reconstructing from this repo gives you exactly the checkpoint that was
trained, not an approximation of it.
Loading
The modeling code imports the dms package, so install it first:
git clone https://github.com/NVIDIA/Model-Optimizer
cd Model-Optimizer/experimental/dms && pip install -e .
Then, from a clone of this repo:
python load_dms.py --out ./dms_student # pulls Qwen/Qwen3-8B, applies the adapters, saves
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("./dms_student", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("./dms_student")
AutoModelForCausalLM.from_pretrained("<this repo>", trust_remote_code=True) will not work
directly: there are no base weights here, and trust_remote_code resolves auto_map against the
repo being loaded rather than the one the config came from. load_dms.py handles both issues; see
its docstring for the detail.
Training
| base | Qwen/Qwen3-8B (frozen) |
| target compression | final_cr = 4.0 (dms_cr = 1/(1-closed_frac)) |
| context / sliding window | 8192 / 512 |
| data | OpenR1-Math-220k, 4000 samples |
| steps | 544 (CR ramp over the first 510) |
| effective batch | 16 |
| optimizer | AdamW, lr 3e-5 constant, beta2 0.95, bf16 |
| hardware / wall clock | 2x H200, 28.7 h |
The full config is in train_config.yaml. Objective is KL distillation against the frozen base as
teacher, plus a compression hinge max(0, target_frac - closed_frac).
Measured at the end of training
| metric | last step | mean of last 32 steps |
|---|---|---|
dms_cr (achieved compression) | 4.514 | 4.198 |
dms_target_cr | 4.0 | 4.0 |
| distillation loss (KL vs base) | 0.00431 | 0.00377 |
| detached LM loss | 0.4866 | 0.4668 |
The compression target is met. These adapters have not been benchmarked on downstream tasks — no LongBench, RULER, or math-reasoning numbers are claimed here. Treat the KL figure as a training diagnostic, not as evidence of task-level quality.
License
Apache-2.0, matching both Qwen/Qwen3-8B and NVIDIA Model-Optimizer.
- Library
- transformers
- Type
- qwen3
- License
- apache-2.0
- Created
- Aug 27, 2026
- Updated
- Aug 27, 2026