Downloads · 30 days
12
7% of all-time downloads
Nishef/SmolLM2-360M-Full_KNOWLEDGE_RETAINING_ENHANCED_KTO_20251227_151509
SmolLM2-360M-Full_KNOWLEDGE_RETAINING_ENHANCED_KTO_20251227_151509 is a text generation model from Nishef. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
<div align="center" <img src="thesisplots/prospecttheoryvisualization.png" alt="Prospect Theory" width="600"/ </div
Downloads · 30 days
12
7% of all-time downloads
All-time downloads
184
Public
Repo size
17.1 MB
Likes
0
Public
Click a slice to open those files.
.safetensors13.1 MB · 60%
From the Hugging Face model README
This model was fine-tuned using Knowledge-Retaining-Enhanced-KTO - a novel alignment method that combines:
Unlike standard preference optimization methods (DPO, PPO), Knowledge-Retaining-Enhanced-KTO:
| Metric | Value |
|---|---|
| Improvement | 18.7% |
| Training Steps | 416 |
| Base Model | HuggingFaceTB/SmolLM2-360M |
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-360M")
tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-360M")
model = PeftModel.from_pretrained(base_model, "Nishef/SmolLM2-360M-Full_KNOWLEDGE_RETAINING_ENHANCED_KTO_20251227_151509")
prompt = "What is machine learning?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
| Parameter | Value |
|---|---|
| Learning Rate | 2e-4 |
| Batch Size | 32 (effective) |
| LoRA Rank | 64 |
| LoRA Alpha | 128 |
| Epochs | 1 |
| Dataset | trl-lib/ultrafeedback_binarized |
| Method | HellaSwag | TruthfulQA | MMLU | Average |
|---|---|---|---|---|
| DPO | 0.550 | 0.361 | 0.264 | 0.392 |
| ORPO | 0.526 | 0.373 | 0.249 | 0.383 |
| Enhanced KTO | 0.496 | 0.390 | 0.289 | 0.392 |
| Standard KTO | 0.394 | 0.474 | 0.254 | 0.374 |
| Knowledge-Retaining-Enhanced-KTO | 0.392 | 0.450 | 0.244 | 0.362 |
🎯 TruthfulQA Excellence: Our method achieves 0.450 accuracy on TruthfulQA, significantly outperforming DPO (0.361) and ORPO (0.373). This demonstrates the effectiveness of Prospect Theory's loss aversion in promoting truthful outputs.
📈 Comparison with Standard KTO: Knowledge-Retaining-Enhanced-KTO maintains similar TruthfulQA performance (0.450 vs 0.474) while providing more stable training dynamics.
<div align="center"> <img src="thesis_plots/benchmark_comparison.png" alt="Benchmark Comparison" width="700"/> </div>