Downloads · 30 days
19
12% of all-time downloads
Qwen3.5-9B-Base-Nemotron-SFT-5166-steps
ZhuofengLi/Qwen3.5-9B-Base-Nemotron-SFT-5166-steps
Qwen3.5-9B-Base-Nemotron-SFT-5166-steps is a machine learning model from ZhuofengLi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This model is Qwen/Qwen3.5-9B-Base fine-tuned with full-parameter SFT on the NVIDIA Nemotron-Terminal-Corpus, following the training recipe from the paper On Data Engineering for Scaling LLM Terminal Capabilities (NVI…
All-time downloads
163
Public
Parameters
1.5M
18.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors18.8 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- qwen3_5
- Access
- Public
- Created
- Jun 19, 2026
- Updated
- Jun 19, 2026
- SHA
- 84a9eca7
Base models
Datasets
Model card
From the Hugging Face model README
Qwen3.5-9B-Base-Nemotron-SFT-5166-steps
Model Description
This model is Qwen/Qwen3.5-9B-Base fine-tuned with full-parameter SFT on the NVIDIA Nemotron-Terminal-Corpus, following the training recipe from the paper On Data Engineering for Scaling LLM Terminal Capabilities (NVIDIA, 2026). This checkpoint corresponds to step 5166 (final checkpoint after 2 epochs).
Training Data
- Dataset: nvidia/Nemotron-Terminal-Corpus
- Description: ~366k terminal/coding instruction samples curated by NVIDIA to improve LLM capability on shell, CLI, and code tasks.
Training Details
Framework
- Training Framework: ms-swift
- Backend: DeepSpeed ZeRO-3
Hyperparameters
| Parameter | Value |
|---|---|
| Learning Rate | 2e-5 |
| Weight Decay | 1e-4 |
| Epochs | 2 |
| Global Batch Size | 128 |
| Micro-batch Size per GPU | 1 |
| Gradient Accumulation Steps | 2 |
| Max Sequence Length | 32,768 tokens |
| Optimizer | AdamW (β₁=0.9, β₂=0.95) |
| LR Scheduler | Cosine with 10% warmup |
| Gradient Clipping | 1.0 |
| Attention | FlashAttention-2 |
| dtype | bfloat16 |
Hardware
| Item | Value |
|---|---|
| GPU | H200 |
| Nodes | 8 |
| GPUs per Node | 8 |
| Total GPUs | 64 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"ZhuofengLi/Qwen3.5-9B-Base-Nemotron-SFT-5166-steps",
torch_dtype="auto",
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("ZhuofengLi/Qwen3.5-9B-Base-Nemotron-SFT-5166-steps")
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Jun 19, 2026
- Updated
- Jun 19, 2026