Downloads · 30 days
26
62% of all-time downloads
WaltonFuture/fast-dllm-v2-1p5b-pumam-am-cascade-if-4k-6000step
fast-dllm-v2-1p5b-pumam-am-cascade-if-4k-6000step is a machine learning model from WaltonFuture. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for lmflow. The card lists the license as apache-2.0.
Block-diffusion (dLLM) model fine-tuned with PUMA-M policy, evaluated with PUMA-G reveal.
Downloads · 30 days
26
62% of all-time downloads
All-time downloads
42
Public
Parameters
1.5B
43.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.pt37.1 GB · 86%
From the Hugging Face model README
Block-diffusion (dLLM) model fine-tuned with PUMA-M policy, evaluated with PUMA-G reveal.
Fast-dLLM v2 1.5B AR-init converted from Qwen/Qwen2.5-1.5B-Instruct
(mask token |<MASK>|, id 151665).
Chat-template tokenized length ≤ 4096 (6,205,273 of 6,861,914 rows kept):
| Source | Rows (before 4K filter) |
|---|---|
| a-m-team/AM-Thinking-v1-Distilled | 1,890,607 |
| nvidia/Nemotron-Cascade-SFT-Stage-1 general.jsonl | 1,171,104 |
| nvidia/Nemotron-Cascade-SFT-Stage-2 general/*.jsonl | 3,550,455 |
| nvidia/Nemotron-SFT-Instruction-Following-Chat-v3 | 249,748 |
| Item | Value |
|---|---|
| policy | SBD PUMA-M (sbd_puma_policy=m), 2 forwards |
| steps | 6000 (warmup 100, linear decay) |
| global batch | 2 per-device × 16 grad-accum × 8 GPUs = 256 |
| block size | 4096 (max full token length assert 4096) |
| learning rate | 2e-5 |
| precision | bf16, DeepSpeed ZeRO-3, gradient checkpointing |
| seed | 42 |
| save | every 500 steps (this upload = checkpoint-6000) |
sbd_reveal_policy=g)| Benchmark | Metric | Score |
|---|---|---|
| MMLU (5-shot) | acc | 49.90 |
| GPQA main | acc | 27.90 |
| GSM8K (0-shot) | exact_match (flexible-extract) | 60.96 |
| Minerva Math (0-shot) | math_verify | 31.08 |
| Minerva Math (0-shot) | exact_match | 0.00 (format mismatch: model writes Final Answer**: \(\boxed{...}\) instead of the Minerva fixed closing phrase; see math_verify for the semantic score) |
| IFEval | inst_level_strict / loose | 64.51 / 67.87 |
| IFEval | prompt_level_strict / loose | 53.60 / 57.67 |
Minerva Math math_verify by subtask: prealgebra 47.19, algebra 45.58, counting_and_prob 25.74, geometry 24.22, num_theory 22.78, precalc 18.50, intermediate_algebra 15.50.
Repo root = checkpoint-6000 (the final model evaluated above):
model.safetensors + config/tokenizer: everything needed for inference/evalglobal_step6000/, latest, scheduler.pt, rng_state_*.pth, trainer_state.json,
training_args.bin, sbd_adaptive_state.pt: DeepSpeed ZeRO-3 resume state
(only needed to continue training from step 6000; use zero_to_fp32.py if you
need to rebuild full fp32 weights from shards)Subfolder checkpoint-1500/ = the earliest surviving intermediate checkpoint
(step 1000/500 were pruned by save_total_limit=10; remaining local checkpoints:
1500–6000 every 500 steps). Same layout, resume-from-step-1500 capable.
Evaluate with the Fast-dLLM/lmflow stack: --model fast_dllm_v2 --model_args model_path=<this dir>,is_sbd=True,sbd_reveal_policy=g --apply_chat_template --fewshot_as_multiturn.