Downloads · 30 days
0
Mahesh111000/hanabi-qwen3-32b-lora-step70
hanabi-qwen3-32b-lora-step70 is a machine learning model from Mahesh111000. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
RL training state for resuming on another machine. Base model: Qwen/Qwen3-32B (bf16).
Downloads · 30 days
0
Access
Public
Updated Jul 22, 2026
Repo size
1.1 GB
Likes
0
Public
Click a slice to open those files.
.safetensors1.1 GB · 100%
From the Hugging Face model README
RL training state for resuming on another machine. Base model: Qwen/Qwen3-32B (bf16).
Note: 97.5% of
checkpoint_step_70/is the frozen base model stored as fp32 master weights (32.76B x 4 bytes = 131 GB). Only ~3.2 GB is unique. If you don't need the exact optimizer state and step counter, just useadapter/(1.07 GB) with a fresh optimizer — it holds all the learning, and it lets you change the learning rate (a resume does not).
| path | size | purpose |
|---|---|---|
adapter/ | 1.1 GB | LoRA adapter (r=32, alpha=64). The entire trained delta. PEFT/vLLM loadable. |
checkpoint_step_70/ | 126 GB | prime-rl DCP checkpoint — model + optimizer + scheduler, exact resume. |
code/ | small | the custom reward env, config, and launcher (not on PyPI — required). |
max_tokens 26624, seq_len 32768.deduction_score + move_reward (range [0,2]).| step | reward | grad norm | entropy | mismatch KL |
|---|---|---|---|---|
| 65 | 1.4244 | 0.0008 | 0.5994 | 0.0029 |
| 66 | 1.3072 | 0.0007 | 0.5936 | 0.0029 |
| 67 | 1.4862 | 0.0007 | 0.6049 | 0.0030 |
| 68 | 1.4189 | 0.0009 | 0.6000 | 0.0030 |
| 69 | 1.3700 | — | — | — |
Reward is flat within noise over this window — no trend either way. is_truncated = 0.0
throughout, so nothing is hitting the 26,624-token cap. copy_rate ~0.55 and flat, i.e. the
degenerate "copy the previous belief state" policy is not being learned.
The small grad norm is expected, not a fault: the loss is token-mean-reduced over ~3.9M tokens per step and advantages are raw within-group deviations (no std normalisation), while Adam is scale-invariant so updates remain ~lr-sized.
git clone https://github.com/PrimeIntellect-ai/prime-rl.git
cd prime-rl && git checkout 16e747c25e81c5c90bd860c68a588f7b151353a6
uv sync
huggingface-cli download Mahesh111000/hanabi-qwen3-32b-lora-step70 --local-dir ./hanabi-state
# custom reward env (required — not on PyPI)
uv pip install --python .venv/bin/python --no-deps -e ./hanabi-state/code/hanabi-deduction
cp ./hanabi-state/code/rl_32b.toml configs/hanabi_deduction/
mkdir -p outputs/checkpoints
cp -r ./hanabi-state/checkpoint_step_70 outputs/checkpoints/step_70
VLLM_ATTENTION_BACKEND=FLASHINFER \
uv run rl @ configs/hanabi_deduction/rl_32b.toml --ckpt.resume-step 70
Dataset: Mahesh111000/Hanabi-init-30turns (or point dataset_path at a local copy).
Expect: the first step after any (re)start is a cold start while the async pipeline refills
(Async Level climbs from -2 to 0) — measured 104 min, then 88, then settling to ~74–88 min
per step. The trainer needs ~40 min and hides entirely inside the rollout phase, so step time is
inference-bound. Needs 8x80 GB GPUs.
Full code, configs and profiling tooling: https://github.com/kaousheik-26/hanabi_latest_train_32b
dist_timeout_seconds = 18000 — the default 600s kills the trainer on the cold-start step.[orchestrator.client] timeout = 7200 — the default 1200s silently drops long rollouts.ckpt.keep_last = 2 — without it, checkpoints fill the disk (~126 GB per save).[trainer.tokenizer] name pinned — prime-rl otherwise resolves it from a stale default.lr is ignored on resume. To
change the learning rate you must start fresh from merged weights, not resume.outputs/weights/ are NOT merged (bitwise identical to base) — don't rely on
them. clean_lora_state_dict() (trainer/lora.py:220) discards the adapter. Use merge_lora.py.step_N is the state to resume at N, i.e. saved after step N-1 finished.verifiers sets prompt_logprobs=True, which makes vLLM
set skip_reading_prefix_cache and bypass the cache entirely (0 queries, not 0 hits).