Downloads · 30 days
470
100% of all-time downloads
dlm-code0.6b-exp
staticlabs/dlm-code0.6b-exp
dlm-code0.6b-exp is a text generation model from staticlabs. Use it when you need the model to write or continue text.
Experimental masked-diffusion language model for code, from the thunder-fast project.
All-time downloads
470
Public
Parameters
630M
1.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.3 GB · 99%
At a glance
- Task
- Text Generation
- Model type
- qwen2
- Access
- Public
- Created
- Sep 3, 2026
- Updated
- Sep 3, 2026
- SHA
- bc8ba20a
Try a prompt
Model card
From the Hugging Face model README
dlm-code0.6b-exp
Experimental masked-diffusion language model for code, from the thunder-fast project.
This is a discrete masked-diffusion (MDM) adaptation of a Qwen2-based pretrained code model (~0.5B). It is not an autoregressive next-token model: generation is a progressive un-masking loop under bidirectional attention, so a whole output window is filled in parallel over 24 diffusion steps.
Details
| Backbone | Qwen2 (model_type: qwen2), 24 layers, hidden 896, 14 attention / 2 KV heads, GQA |
| Vocabulary | 151,936 + <M> mask token (id 151665) |
| Diffusion | discrete [MASK], 24-step progressive un-masking, entropy-based position confidence (alg_temp 0.6, top_k 500) |
| Context | window-based (256 tokens) — not left-to-right; supports infilling |
| Weights | bf16 (model.safetensors) |
How to run
This checkpoint is intended to be driven by a diffusion decoder (bidirectional attention + the iterative un-masking loop), e.g. the thunder-fast runtime. Loading it as a plain causal Qwen2ForCausalLM will not reproduce diffusion generation.
# thunder-fast runtime (ggml engine) or the reference DiffusionLM
# e.g. runtime --model dlm-code0.6b-exp.gguf --prompt "..."
Status
Experimental / work-in-progress (.exp). It is a diffusion code-generation checkpoint published for the thunder-fast project; see the repo for training, the masking schedule, and evaluation details.
- Task
- Text Generation
- Type
- qwen2
- Languages
- en
- Created
- Sep 3, 2026
- Updated
- Sep 3, 2026