Downloads · 30 days
26
5% of all-time downloads
small-1b-dpo
kenpeter123/small-1b-dpo
small-1b-dpo is a text generation model from kenpeter123. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
DPO (Direct Preference Optimization) model — 1B Transformer++ base, post-trained on top of SFT (kenpeter123/small-1b-sft) using preference pairs. Reference model = policy at init (standard DPO from the SFT anchor). 8-…
All-time downloads
482
Public
Parameters
1B
4.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt2.1 GB · 50%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Aug 20, 2026
- Updated
- Aug 20, 2026
- SHA
- d7be3acd
Try a prompt
Base models
Model card
From the Hugging Face model README
Small-1B DPO Model
DPO (Direct Preference Optimization) model — 1B Transformer++ base, post-trained on top of SFT (kenpeter123/small-1b-sft) using preference pairs. Reference model = policy at init (standard DPO from the SFT anchor). 8-bit Adam, bf16, stage-isolated checkpoint.
| Spec | Value |
|---|---|
| Params | 1,031,898,624 (~1.03B) |
| Hidden | 1536 · 32 layers · 12 attn heads · 4 KV (GQA) · head_dim 128 |
| FFN | 4608 SwiGLU |
| Seq len | 2048 (train) / 8192 (config) |
| Vocab | 49152 (SmolLM2-135M tokenizer, BPE) |
| DPO | steps 300 · lr 1e-05 · β 0.1 · sigmoid loss |
| Optimizer | bitsandbytes Adam8bit, bf16 |
Preference pairs: raw problem prompt (chosen = gold correct solution, rejected = SFT model's
wrong/echo generation). Raw training checkpoint included as dpo_best.pt
(model_state_dict only, bf16).
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Aug 20, 2026
- Updated
- Aug 20, 2026