Downloads · 30 days
9
23% of all-time downloads
LLM2026_DPO_SFT19_v9
makotonlo/LLM2026_DPO_SFT19_v9
LLM2026_DPO_SFT19_v9 is a text generation model from makotonlo. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
This model is a LoRA adapter fine-tuned from unsloth/Qwen2.5-7B-Instruct-bnb-4bit using Direct Preference Optimization (DPO). The base weights for training were from SFT model: makotonlo/LLM2026SFTfinalv197B.
All-time downloads
40
Public
Repo size
646 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors646 MB · 100%
At a glance
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Feb 22, 2026
- Updated
- Feb 22, 2026
- SHA
- 748da193
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
LLM2026_DPO_SFT19_v9
This model is a LoRA adapter fine-tuned from unsloth/Qwen2.5-7B-Instruct-bnb-4bit using Direct Preference Optimization (DPO). The base weights for training were from SFT model: makotonlo/LLM2026_SFT_finalv19_7B.
Training Objective
Optimized for high-quality JSON output and logical reasoning through DPO.
Training Configuration
- Base model: unsloth/Qwen2.5-7B-Instruct-bnb-4bit
- Method: DPO (Direct Preference Optimization)
- Max Steps: 500
- Learning rate: 1e-05
- Beta: 0.1
- LoRA Config: r=64, alpha=64
Usage
This is a LoRA adapter. You can load it using unsloth or vLLM by pointing to this repository.
Framework versions
- PEFT 0.13.2
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Languages
- en
- Created
- Feb 22, 2026
- Updated
- Feb 22, 2026