Downloads · 30 days
0
haotiansun014/qwen3-4b-stage3-nce-7335-temp07-archive
qwen3-4b-stage3-nce-7335-temp07-archive is a machine learning model from haotiansun014. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Stage-3 Noise-Contrastive-Estimation training resumed from the Stage-2 end ckpt at step 7335. NCE phase trains the scorer head to rank K=4 candidate completions per block. Differs from beam-perround in proposal sampli…
Downloads · 30 days
0
Access
Public
Updated May 2, 2026
Repo size
9.5 GB
Likes
0
Public
Click a slice to open those files.
.pt9.5 GB · 100%
From the Hugging Face model README
Stage-3 Noise-Contrastive-Estimation training resumed from the Stage-2
end ckpt at step 7335. NCE phase trains the scorer head to rank K=4
candidate completions per block. Differs from beam-perround in proposal
sampling: this variant uses softmax_sampling at proposal_temperature
=0.7 (vs argmax in beam-perround) — closer to training-time scorer
noise.
| File | Size |
|---|---|
model.pt | 8.5 GB |
optimizer.pt | 0.45 GB |
scheduler.pt | 1.7 KB |
eval_batches.pt | 13 MB |
rng_rank{0..23}.pt | 14.7 KB each |
Total: ~9.54 GB / 30 files. Full resume state for re-training.
combine=softmax_sampling, proposal_temperature=0.7configs/large_scale/qwen3_4b_stage3_nce_resume7335_temp07_perround_6n.yamlWhen evaluating this ckpt, use:
combine=softmax_sampling proposal_temperature=0.7
to MATCH the training distribution. Using combine=beam_bayes argmax
(as for the perround/lastround variants) on this ckpt produces an
out-of-distribution scorer signal.
haotiansun014/qwen3-4b-stage3-nce-7335-perround-archivehaotiansun014/qwen3-4b-stage3-nce-7335-lastround-archive