Downloads · 30 days
6
20% of all-time downloads
SeanWang0027/qwen3-4b-async-rose-polaris-k4096-step20
qwen3-4b-async-rose-polaris-k4096-step20 is a text generation model from SeanWang0027. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is a 20-step intermediate checkpoint. Its effect is unverified. Read the "What this is not" section before using it for anything.
Downloads · 30 days
6
20% of all-time downloads
All-time downloads
30
Public
Parameters
4.4B
33.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.pt24.9 GB · 74%
From the Hugging Face model README
This is a 20-step intermediate checkpoint. Its effect is unverified. Read the "What this is not" section before using it for anything.
| student (base) | Qwen/Qwen3-4B |
| teacher | Qwen/Qwen3-32B, local vLLM, --return-tokens-as-token-ids |
| data | SeanWang0027/polaris_hard — 15368 competition maths problems |
| objective | online ROSE, policy_loss.loss_mode=ce |
| cut | K=4096 student prefix + t=1024 teacher tokens → max_response_length 5120 |
| schedule | asynchronous — recipe/async_rose_online, depth=3, max_age=8 |
| steps | 20 of ~240 in one epoch (6.7%) |
| batch | train_batch_size=64, n=4 → 256 responses per update |
| optimiser | AdamW, lr 1e-5 constant, no warmup, betas (0.9, 0.999), wd 0.01, clip 1.0 |
| precision | bf16, FSDP full shard, gradient checkpointing |
| hardware | 4 × A100-SXM4-80GB — 2 trainer + 2 teacher, ~565 s/step, 3h24m total |
Each row: the student samples a 4096-token prefix, the teacher continues it for up to 1024 tokens, and cross-entropy is taken only over the teacher's tokens (19.2% of the sequence). The prefix carries no gradient.
Student and teacher share a byte-identical tokenizer.json (sha256
aeb13307a71acd8f…, vocab 151936), so the teacher continues from the student's
raw token ids with no text round trip and no seam correction.
ce_loss showed no trend across the 20 steps — 0.612 → 0.559 → … → 0.615,
oscillating in 0.55–0.62. grad_norm fell monotonically, 3.61 → 0.396.compute_policy_loss_ce ignores
advantages; the polaris reward exists to make logs legible and, in this
recipe, is not even logged.Treat this as a pipeline artifact, not a model release.
./ merged HF weights — from_pretrained works directly
fsdp_step20/ verl FSDP checkpoint, for resuming training
fsdp_step20/ holds model_world_size_2_rank_{0,1}.pt,
optim_world_size_2_rank_{0,1}.pt and extra_state_*. Two caveats:
world_size=2. Resuming needs the same world size
and FSDP config; 1 or 4 ranks require a reshard first.exp_avg and exp_avg_sq are both
torch.bfloat16. There is no separate fp32 master copy in the checkpoint.lm_head.weight is present in the merged weights and is byte-identical to
model.embed_tokens.weight; tie_word_embeddings is true, so it is re-tied on
load and the duplicate is inert.
python3 scripts/prepare_polaris.py
MODEL_DIR=/root/models TEACHER_GPUS=0,1 bash scripts/run_teacher_32b.sh
MODEL_DIR=/root/models TRAINER_GPUS=2,3 \
CUT_FIXED=4096 TEACHER_MAX_TOKENS=1024 DEPTH=3 TOTAL_STEPS=20 SAVE_FREQ=20 \
bash scripts/run_polaris_qwen3_4b_qwen32b_k1024_t1024_async.sh
Environment that matters: vllm==0.11.0, transformers==4.57.1 (not 5.x),
flash-attn==2.8.3, torch 2.8.0+cu128.