Downloads · 30 days
6
5% of all-time downloads
4B-Instruct-distill-jx739p0v-step4724
JackHsieh/4B-Instruct-distill-jx739p0v-step4724
4B-Instruct-distill-jx739p0v-step4724 is a machine learning model from JackHsieh. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Qwen3-4B-Instruct-2507 distilled on gpt-5.6-luna thoughts about the next 8 Qwen3 tokens of stat.ML arXiv LaTeX.
All-time downloads
123
Public
Parameters
4.4B
17.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors17.6 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- qwen3
- Access
- Public
- Created
- Aug 30, 2026
- Updated
- Aug 30, 2026
- SHA
- 0838b8ef
Base models
Model card
From the Hugging Face model README
4B-Instruct-distill-jx739p0v-step4724
Qwen3-4B-Instruct-2507 distilled on gpt-5.6-luna thoughts about the next 8 Qwen3 tokens of
stat.ML arXiv LaTeX.
SFT: 2 epochs over 604,766 examples, lr 3e-5 cosine, batch 256. Val log prob on held-out luna thoughts at this checkpoint: -0.8691 nats/token (from -3.3492 at step 0). This is the last saved step of a 4-epoch run that stopped early; step 4724 is exactly the end of epoch 2.
Weights are stored as fp32 master weights (config.json declares bfloat16); load with
dtype=torch.bfloat16 to match how the model is used downstream.
Prompt it with the SFT chat template shipped here as chat_template.jinja — luna's own
v5trim-qwen system prompt minus its few-shot exchanges. Generation stops on <|im_end|>.
Recommended sampling (Qwen3-2507): temperature 0.7, top_p 0.8, top_k 20, min_p 0.
- Type
- qwen3
- License
- apache-2.0
- Created
- Aug 30, 2026
- Updated
- Aug 30, 2026