Downloads · 30 days
15
38% of all-time downloads
ZhuofengLi/qwen3.5-9b-nemotron-sft-ckpt150
qwen3.5-9b-nemotron-sft-ckpt150 is a machine learning model from ZhuofengLi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Intermediate SFT checkpoint (step 150/386, epoch ~0.78/2) of Qwen3.5-9B fine-tuned on the NVIDIA Nemotron-Terminal-Corpus dataset using ms-swift with DeepSpeed ZeRO-3. Trained to improve terminal interaction and agent…
Downloads · 30 days
15
38% of all-time downloads
All-time downloads
39
Public
Parameters
1.5M
18.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors18.8 GB · 100%
From the Hugging Face model README
Intermediate SFT checkpoint (step 150/386, epoch ~0.78/2) of Qwen3.5-9B fine-tuned on the NVIDIA Nemotron-Terminal-Corpus dataset using ms-swift with DeepSpeed ZeRO-3. Trained to improve terminal interaction and agentic capabilities. Training loss at this checkpoint: 0.3244.
| Parameter | Value |
|---|---|
| Learning Rate | 2e-5 |
| Weight Decay | 1e-4 |
| Epochs | 2 |
| Global Batch Size | 64 (per_device=1, grad_accum=2, 32 DP) |
| Micro-batch Size per GPU | 1 |
| Max Sequence Length | 262144 tokens |
| Optimizer | AdamW (β₁=0.9, β₂=0.95) |
| LR Scheduler | Cosine with 10% warmup |
| Gradient Clipping | 1.0 |
| Packing | Enabled |
| Precision | BF16 |
| Item | Value |
|---|---|
| GPU | H200 |
| Nodes | 8 |
| GPUs per Node | 8 |
| Total GPUs | 64 |
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(ZhuofengLi/qwen3.5-9b-nemotron-sft-ckpt150, torch_dtype=auto)
tokenizer = AutoTokenizer.from_pretrained(ZhuofengLi/qwen3.5-9b-nemotron-sft-ckpt150)