Downloads · 30 days
5
2% of all-time downloads
SeanWang0027/student_prefix_polaris_15K_qwen3-4b_continual_Q_qwen3-32b_cutoff4096_epoch_1_mask
student_prefix_polaris_15K_qwen3-4b_continual_Q_qwen3-32b_cutoff4096_epoch_1_mask is a text generation model from SeanWang0027. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Online ROSE. Qwen3-4B (student) trained on the continuations that Qwen3-32B (teacher) writes from the student's own 4096-token prefix, with cross-entropy applied only to the teacher's tokens.
Downloads · 30 days
5
2% of all-time downloads
All-time downloads
308
Public
Parameters
4.4B
8.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8.8 GB · 100%
From the Hugging Face model README
Online ROSE. Qwen3-4B (student) trained on the continuations that Qwen3-32B (teacher) writes from the student's own 4096-token prefix, with cross-entropy applied only to the teacher's tokens.
prompt ---- student prefix (4096 tok) ------- teacher continuation (1024 tok) ----
response_mask 0 response_mask 1
context, on-policy the target, off-policy
| student | Qwen/Qwen3-4B |
| teacher | Qwen/Qwen3-32B (same tokenizer, byte-identical tokenizer.json) |
| data | SeanWang0027/polaris_hard, 15,368 competition-maths problems |
| cut | K = 4096 prefix + t = 1024 teacher tokens → max_response_length 5120 |
| schedule | asynchronous (recipe/async_rose_online), depth 3 |
| batch | train_batch_size 64, n 4 → 256 sequences per block |
| optimizer | AdamW, lr 1e-5 constant, no warmup, wd 0.01, grad clip 1.0 |
| objective | ce — masked NLL over the teacher's tokens only |
| steps | 240 (one epoch), global_step_240 |
| hardware | 8 × NVIDIA GH200 on TACC Vista — 4 teacher replicas (vLLM TP=1) + 4 trainer ranks (FSDP) |
| wall clock | 10 h 52 m (163 s/step) |
Code: cl-from-nothing/online-rose,
docs/POLARIS_QWEN3_4B_QWEN32B_K4096_8GPU.md.
The objective did not move over the 240 steps, and the weights barely changed. These are measured, not estimated:
actor/ce_nll_before, mean of first 20 steps | 0.6010 |
actor/ce_nll_before, mean of last 20 steps | 0.6026 (+0.3%) |
| range across all 240 steps | 0.5454 – 0.6748 (noise) |
parameters that changed at all vs Qwen/Qwen3-4B | 12.32% (543.7 M / 4.41 B) |
| global relative L2 drift vs base | 1.75e-3 |
| largest per-tensor relative drift | 3.36e-3 (layers.34.self_attn.k_proj) |
The most likely explanation is the training precision. The launcher passes
actor_rollout_ref.actor.fsdp_config.model_dtype=bfloat16, which overrides the
torch.float32 that verl otherwise forces for the actor — and verl's own comment at
fsdp_workers.py:302 says "we have to create model in fp32. Otherwise, the optimizer
is in bf16, which is incorrect." With bf16 master weights and bf16 Adam moments, one
step at lr = 1e-5 is the same order as a bf16 half-ULP at |w| ≈ 0.02, so most weights
cannot move. That 87.7% of parameters are bit-identical to the base is consistent with
this and hard to explain otherwise.
No downstream evaluation has been run. Neither this checkpoint nor the base model
has been measured on AIME 2025 or anything else. The reward logged during training
(critic/score/mean ≈ 0.06–0.14) is log-only — compute_policy_loss_ce ignores
advantages — and it scores a truncated prefix spliced to a partial continuation, so
it says nothing about the student.
Treat this as a reproduction artifact of the training run, not a model claimed to be
better than Qwen/Qwen3-4B. If you want the comparison, run both under identical
settings.
| metric | value | note |
|---|---|---|
rose/pool_failures | 0 across all 240 steps | no teacher request ever failed |
rose/prefix_age | pinned at 2.0 = depth − 1 | the async pipeline behaved as designed |
rose/teacher_hidden_frac | ≈ 0.84 steady | 16% of teacher latency not hidden |
rose/empty | ≈ 3% mean (max 12%) | rows dropped by on_early_finish=drop |
rose/teacher_finished | ≈ 0.016 | near zero — targets almost never contain a terminator, so the student is taught to continue and never to stop |
rose/teacher_finished near zero is a property of this K/t choice, not a bug: after a
4096-token prefix, 1024 more tokens rarely reach the end of a competition-maths solution.
from transformers import AutoModelForCausalLM, AutoTokenizer
mid = "SeanWang0027/student_prefix_polaris_15K_qwen3-4b_continual_Q_qwen3-32b_cutoff4096_epoch_1_mask"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype="bfloat16", device_map="auto")