Downloads · 30 days
9
100% of all-time downloads
nagendra-sahaj/llama-3.2-3b-r1-distill-sft
llama-3.2-3b-r1-distill-sft is a machine learning model from nagendra-sahaj. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as llama3.2.
A fine-tuned version of meta-llama/Llama-3.2-3B-Instruct trained on the ServiceNow-AI/R1-Distill-SFT dataset to transfer chain-of-thought reasoning capabilities into a smaller, efficient 3B model.
Downloads · 30 days
9
100% of all-time downloads
All-time downloads
9
Public
Parameters
3.2B
6.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors6.4 GB · 100%
From the Hugging Face model README
A fine-tuned version of meta-llama/Llama-3.2-3B-Instruct trained on the ServiceNow-AI/R1-Distill-SFT dataset to transfer chain-of-thought reasoning capabilities into a smaller, efficient 3B model.
This model was fine-tuned using Supervised Fine-Tuning (SFT) with QLoRA (4-bit quantization) on math reasoning problems. The training dataset was created by ServiceNow's SLAM Lab, where DeepSeek-R1-32B was used as a teacher model to generate detailed chain-of-thought reasoning responses for ~171K math problems sourced from NuminaMath and Tulu.
The goal is to distill the structured reasoning style of a large teacher model into a much smaller 3B model that is cheaper and faster to serve.
<think> reasoning chains before answering| Parameter | Value |
|---|---|
| Base model | meta-llama/Llama-3.2-3B-Instruct |
| Dataset | ServiceNow-AI/R1-Distill-SFT (v0) |
| Dataset size | ~130K rows (after length filtering) |
| Training steps | 500 / ~10,154 (~5% of full training) |
| Method | SFT + QLoRA (4-bit) |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| Sequence length | 2048 |
| Batch size | 16 (effective) |
| Learning rate | 2e-4 |
| LR scheduler | Cosine |
| Optimizer | AdamW 8-bit |
| Hardware | AWS EC2 g5.4xlarge (NVIDIA A10G 24GB) |
| Framework | Unsloth + TRL SFTTrainer |
Evaluated on a held-out test set (5% of filtered dataset, ~6,500 rows):
| Metric | Base Model | This Model | Improvement |
|---|---|---|---|
| Avg Loss | 0.7826 | 0.4077 | ⬇️ 47.9% |
| Perplexity | 2.1871 | 1.5034 | ⬇️ 31.3% |
Note: These results are from 500 training steps (~5% of planned full training). Further improvement is expected on completing the full training run.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "nagendra-sahaj/llama-3.2-3b-r1-distill-sft"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
prompt = (
"<|begin_of_text|>"
"<|start_header_id|>user<|end_header_id|>\n"
"Solve for x: 3x + 7 = 22\n"
"<|eot_id|>"
"<|start_header_id|>assistant<|end_header_id|>\n"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.3,
do_sample=True,
repetition_penalty=1.3,
eos_token_id=tokenizer.convert_tokens_to_ids("<|eot_id|>"),
)
print(tokenizer.decode(output[0], skip_special_tokens=True))
<think>
Let me solve this step by step.
We need to find x in: 3x + 7 = 22
First, subtract 7 from both sides:
3x + 7 - 7 = 22 - 7
3x = 15
Then divide both sides by 3:
3x / 3 = 15 / 3
x = 5
</think>
The value of x is 5.
ServiceNow-AI/R1-Distill-SFT — v0 split containing ~171K math problems with reasoning responses generated by DeepSeek-R1-32B. After filtering responses longer than 1500 words, ~130K rows were used.
If you use this model, please cite the original dataset:
@misc{slam-distillation-from-r1,
author = {Sathwik Tejaswi Madhusudhan and Shruthan Radhakrishna and Jash Mehta and Toby Liang},
title = {Millions scale dataset distilled from R1-32b},
publisher = {SLAM - ServiceNow Language Models Lab},
year = {2025},
url = {https://huggingface.co/datasets/ServiceNow-AI/R1-Distill-SFT}
}