Downloads · 30 days
0
Mahesh111000/hanabi-qwen3-32b-lora-step75
hanabi-qwen3-32b-lora-step75 is a machine learning model from Mahesh111000. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
RL training state for resuming on another machine. Base model: Qwen/Qwen3-32B (bf16).
Downloads · 30 days
0
Access
Public
Updated Jul 22, 2026
Repo size
135 GB
Likes
0
Public
Click a slice to open those files.
.distcp134 GB · 99%
From the Hugging Face model README
RL training state for resuming on another machine. Base model: Qwen/Qwen3-32B (bf16).
Note: 97.5% of
checkpoint_step_75/is the frozen base model stored as fp32 master weights (32.76B x 4 bytes = 131 GB). Only ~3.2 GB is unique. If you don't need the exact optimizer state and step counter, just useadapter/(1.07 GB) with a fresh optimizer — it holds all the learning, and it lets you change the learning rate (a resume does not).
| path | size | purpose |
|---|---|---|
adapter/ | 1.1 GB | LoRA adapter (r=32, alpha=64). The entire trained delta. PEFT/vLLM loadable. |
checkpoint_step_75/ | 126 GB | prime-rl DCP checkpoint — model + optimizer + scheduler, exact resume. |
code/ | small | the custom reward env, config, and launcher (not on PyPI — required). |
max_tokens 26624, seq_len 32768.deduction_score + move_reward (range [0,2]).| step | reward | grad norm | entropy | mismatch KL |
|---|---|---|---|---|
| 65 | 1.4244 | 0.0008 | 0.5994 | 0.0029 |
| 66 | 1.3072 | 0.0007 | 0.5936 | 0.0029 |
| 67 | 1.4862 | 0.0007 | 0.6049 | 0.0030 |
| 68 | 1.4189 | 0.0009 | 0.6000 | 0.0030 |
| 69 | 1.3700 | 0.0008 | 0.6041 | 0.0032 |
| 70 | 1.4810 | 0.0007 | 0.5889 | 0.0033 |
| 71 | 1.5295 | 0.0007 | 0.5794 | 0.0037 |
| 72 | 1.6249 | 0.0007 | 0.5823 | 0.0040 |
| 73 | 1.6333 | 0.0006 | 0.5819 | 0.0038 |
| 74 | 1.4401 | 0.0007 | 0.5918 | 0.0038 |
| 75 | 1.4628 | 0.0007 | 0.5847 | 0.0039 |
Mean reward over 67–75 is 1.494, against 1.424 at step 65. Treat that as flat, not as progress: the step-to-step spread (1.31–1.63) is far wider than the gap between the endpoints, and a 9-point window at this noise level cannot resolve a trend. Entropy drifts down slightly (0.60 → 0.58) and mismatch KL up (0.0029 → 0.0039), both consistent with a policy that is slowly concentrating — worth watching, not yet a problem.
is_truncated = 0.0 throughout, so nothing is hitting the 26,624-token cap. copy_rate ~0.55
and flat, i.e. the degenerate "copy the previous belief state" policy is not being learned.
The small grad norm is expected, not a fault: the loss is token-mean-reduced over ~3.9M tokens per step and advantages are raw within-group deviations (no std normalisation), while Adam is scale-invariant so updates remain ~lr-sized.
git clone https://github.com/PrimeIntellect-ai/prime-rl.git
cd prime-rl && git checkout 16e747c25e81c5c90bd860c68a588f7b151353a6
uv sync
huggingface-cli download Mahesh111000/hanabi-qwen3-32b-lora-step75 --local-dir ./hanabi-state
# custom reward env (required — not on PyPI)
uv pip install --python .venv/bin/python --no-deps -e ./hanabi-state/code/hanabi-deduction
cp ./hanabi-state/code/rl_32b.toml configs/hanabi_deduction/
mkdir -p outputs/checkpoints
cp -r ./hanabi-state/checkpoint_step_75 outputs/checkpoints/step_75
VLLM_ATTENTION_BACKEND=FLASHINFER \
uv run rl @ configs/hanabi_deduction/rl_32b.toml --ckpt.resume-step 75
Dataset: Mahesh111000/Hanabi-init-30turns (or point dataset_path at a local copy).
Expect: the first step after any (re)start is a cold start while the async pipeline refills
(Async Level climbs from -2 to 0) — measured 148 min, then 85, then settling to ~60–92 min
per step. The trainer needs ~40 min and hides entirely inside the rollout phase, so step time is
inference-bound. Needs 8x80 GB GPUs.
VLLM_ATTENTION_BACKEND=FLASHINFER in the command above is not cosmetic — it is the single
biggest win found, measured over full training steps on 8x A100-SXM4-80GB:
| backend | steps | mean min/step |
|---|---|---|
| FlashAttention | 52–66 | 124.5 |
| FlashInfer | 67–75 | 79.7 |
1.56x end-to-end. A standalone decode benchmark predicted only 1.26x; the extra gain comes from the trainer overlapping better once rollouts finish sooner, which a generation-only benchmark cannot see. Since decode here is KV-bandwidth-bound (256 KiB/token, ~5,400-token must-attend prompt = ~47% of KV), attention-kernel efficiency dominates step time.
Full code, configs and profiling tooling: https://github.com/kaousheik-26/hanabi_latest_train_32b
dist_timeout_seconds = 18000 — the default 600s kills the trainer on the cold-start step.[orchestrator.client] timeout = 7200 — the default 1200s silently drops long rollouts.ckpt.keep_last = 2 — without it, checkpoints fill the disk (~126 GB per save).[trainer.tokenizer] name pinned — prime-rl otherwise resolves it from a stale default.lr is ignored on resume. To
change the learning rate you must start fresh from merged weights, not resume.outputs/weights/ are NOT merged (bitwise identical to base) — don't rely on
them. clean_lora_state_dict() (trainer/lora.py:220) discards the adapter. Use merge_lora.py.step_N is the state to resume at N, i.e. saved after step N-1 finished.verifiers sets prompt_logprobs=True, which makes vLLM
set skip_reading_prefix_cache and bypass the cache entirely (0 queries, not 0 hits).