Downloads · 30 days
43
0% of all-time downloads
ahczhg/Llama-3.2-1B-Aegis-SFT-DPO
Llama-3.2-1B-Aegis-SFT-DPO is a text generation model from ahczhg. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.2.
<div align="center" <strongFine-tuned Llama 3.2 1B for Content-Safe Instruction Following</strong </div
Downloads · 30 days
43
0% of all-time downloads
All-time downloads
81.1K
Public
Parameters
1.2B
2.5 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors2.5 GB · 99%
From the Hugging Face model README
This model is a fine-tuned version of meta-llama/Llama-3.2-1B using a two-stage training approach:
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Load model and tokenizer
model_name = "ahczhg/Llama-3.2-1B-Aegis-SFT-DPO"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# Prepare messages
messages = [
{"role": "user", "content": "What is artificial intelligence?"}
]
# Apply chat template and generate
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
outputs = model.generate(
inputs,
max_new_tokens=256,
temperature=0.7,
top_p=0.9,
do_sample=True,
pad_token_id=tokenizer.eos_token_id
)
# Decode response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
This model follows a two-stage approach similar to RLHF (Reinforcement Learning from Human Feedback), inspired by AMD's Instella-3B-Instruct:
Teaching the model to follow the instruction format and generate appropriate responses.
Hyperparameters:
Epochs: 2
Batch Size: 1
Gradient Accumulation: 8
Effective Batch Size: 8
Learning Rate: 1e-5
Optimizer: AdamW
LR Scheduler: Cosine
Warmup Steps: 100
Weight Decay: 0.1
Max Gradient Norm: 1.0
Precision: BF16
Gradient Checkpointing: True
Optimizing the model to prefer safe, helpful responses over problematic ones using preference learning.
Hyperparameters:
Epochs: 1
Batch Size: 1
Gradient Accumulation: 8
Effective Batch Size: 8
Learning Rate: 5e-7
Beta (DPO): 0.1
Max Prompt Length: 512
Max Sequence Length: 1024
Optimizer: AdamW
LR Scheduler: Cosine
Warmup Ratio: 10%
Precision: BF16
Parameter-efficient fine-tuning using Low-Rank Adaptation:
Rank (r): 8
Alpha: 16
Dropout: 0.05
Target Modules:
- q_proj
- k_proj
- v_proj
- o_proj
Bias: none
Task Type: CAUSAL_LM
Trainable Parameters: ~0.5% of total
messages = [
{"role": "user", "content": "What is machine learning?"},
{"role": "assistant", "content": "Machine learning is a subset of AI..."},
{"role": "user", "content": "Can you give me an example?"}
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7, top_p=0.9, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
from transformers import TextIteratorStreamer
from threading import Thread
streamer = TextIteratorStreamer(tokenizer, skip_special_tokens=True)
generation_kwargs = dict(
inputs=inputs,
max_new_tokens=512,
temperature=0.7,
top_p=0.9,
do_sample=True,
streamer=streamer,
pad_token_id=tokenizer.eos_token_id
)
thread = Thread(target=model.generate, kwargs=generation_kwargs)
thread.start()
for new_text in streamer:
print(new_text, end="", flush=True)
thread.join()
prompts = [
"Explain neural networks",
"What is deep learning?",
"How does backpropagation work?"
]
messages_batch = [[{"role": "user", "content": p}] for p in prompts]
# Tokenize all at once
inputs = tokenizer.apply_chat_template(
messages_batch,
add_generation_prompt=True,
return_tensors="pt",
padding=True
).to(model.device)
# Generate
outputs = model.generate(inputs, max_new_tokens=200, temperature=0.7, pad_token_id=tokenizer.eos_token_id)
# Decode all
for output in outputs:
print(tokenizer.decode(output, skip_special_tokens=True))
print("-" * 80)
# More creative
outputs = model.generate(
inputs,
max_new_tokens=512,
temperature=0.9, # Higher = more creative
top_p=0.95,
top_k=50,
do_sample=True,
repetition_penalty=1.1
)
# More focused/deterministic
outputs = model.generate(
inputs,
max_new_tokens=256,
temperature=0.3, # Lower = more focused
top_p=0.85,
do_sample=True,
repetition_penalty=1.05
)
The model uses Llama 3.2's official chat format with special tokens:
<|start_header_id|>user<|end_header_id|>
Your question here<|eot_id|><|start_header_id|>assistant<|end_header_id|>
Model response here<|eot_id|>
The tokenizer's apply_chat_template method handles this automatically.
The model has been tested on:
(Coming soon - add your evaluation results)
| Metric | Base Llama 3.2 | This Model | Improvement |
|---|---|---|---|
| Safety Awareness | Baseline | Enhanced | +Safety Focus |
| Instruction Following | Good | Better | +SFT Training |
| Response Quality | High | High | +DPO Alignment |
This model is licensed under the Llama 3.2 Community License:
@misc{llama_3.2_1b_aegis_sft_dpo,
author = {Community Contributor},
title = {Llama-3.2-1B-Aegis-SFT-DPO: Content-Safe Fine-tuned Llama 3.2},
year = {2024},
publisher = {HuggingFace},
journal = {HuggingFace Model Hub},
howpublished = {\url{https://huggingface.co/ahczhg/Llama-3.2-1B-Aegis-SFT-DPO}}
}
@misc{llama32,
title={Llama 3.2: Open Foundation and Fine-Tuned Chat Models},
author={Meta AI},
year={2024},
url={https://huggingface.co/meta-llama/Llama-3.2-1B}
}
@misc{aegis_dataset,
title={Aegis AI Content Safety Dataset 2.0},
author={NVIDIA},
year={2024},
url={https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0}
}
Found an issue or have suggestions? Please:
Model Card Version: 1.0
Last Updated: 2025-11-15
Training Date: 2025-11-15
Framework Versions:
4.57.12.8.0+cu1260.25.10.17.14.0.0Compute: