Downloads · 30 days
8
24% of all-time downloads
NeelRajani/Qwen3-0.6B-Base_SFT-safety50_D2D-v00.01
Qwen3-0.6B-Base_SFT-safety50_D2D-v00.01 is a text generation model from NeelRajani. Use it when you need the model to write or continue text. It is set up for peft.
Part of the shallowness study (how deep does safety training go?).
Downloads · 30 days
8
24% of all-time downloads
All-time downloads
33
Public
Repo size
254 MB
Likes
0
Public
Click a slice to open those files.
.safetensors80.8 MB · 88%
From the Hugging Face model README
Part of the shallowness study (how deep does safety training go?).
A rank-32 LoRA distilled onto the frozen Qwen/Qwen3-0.6B-Base to reproduce
the output distribution of a safety-SFT teacher — the teacher checkpoint saved at
50% of the safety data
(NeelRajani/Qwen3-0.6B-Base_SFT_safety_v00.01, revision
v00.01-step-000000354).
lmbda=0, beta=0), divergence in fp32all-linear, dropout 0Neelectric/Nemotron-SFT-Safety-v1-nocot, 400 optimizer steps @ effective batch 8recipes/Qwen/Qwen-0.6B-Base/d2d_stage/config_v00.01.yamlOne branch per LoRA rank: v00.01-r32.
from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B-Base")
model = PeftModel.from_pretrained(base, "NeelRajani/Qwen3-0.6B-Base_SFT-safety50_D2D-v00.01", revision="v00.01-r32")