Downloads · 30 days
6
15% of all-time downloads
Mishrakshitij/spar-dpo-lora-adapter-ckpt1000
spar-dpo-lora-adapter-ckpt1000 is a text generation model from Mishrakshitij. Use it when you need the model to write or continue text. It is set up for peft.
DPO LoRA adapter trained on SFT-full Qwen3-4B for debate persuasion, part of the PERSUADE framework (A Debate Arena Framework for Benchmarking, Optimizing, and Ethically Auditing Persuasion in Language Models).
Downloads · 30 days
6
15% of all-time downloads
All-time downloads
41
Public
Repo size
132 MB
Likes
0
Public
Click a slice to open those files.
.safetensors132 MB · 100%
From the Hugging Face model README
DPO LoRA adapter trained on SFT-full Qwen3-4B for debate persuasion, part of the PERSUADE framework (A Debate Arena Framework for Benchmarking, Optimizing, and Ethically Auditing Persuasion in Language Models).
| Base model | Mishrakshitij/spar-qwen3-4b-sft-full (Qwen3-4B-Instruct, fully fine-tuned with SFT on arena demonstrations) |
| Method | DPO (Direct Preference Optimization) with beta=0.2 |
| LoRA config | r=16, alpha=32, dropout=0.05, all linear layers |
| Preference pairs | 81,192 pairs from debate arena |
| Checkpoint | Step 1000 / 10149 |
The preference pairs are derived from a large-scale debate arena where 10 LLMs (ranging from 4B to 235B parameters) debated each other on 259 diverse topics.
For each debate involving the chosen model (Kimi-K2, the top-performing arena model), preference pairs are constructed at the turn level:
chosen and arguments from the losing side become rejected<think>...</think>) tags are stripped from arguments before pairing, so the model learns from clean debate text onlyThis produces 81,192 preference pairs from the training split, where each pair shares the same debate context (topic, position, turn number) but differs in argument quality as determined by the judge's outcome.
| Parameter | Value |
|---|---|
| Learning rate | 5e-6 |
| Beta (DPO) | 0.2 |
| Batch size | 8 |
| Epochs | 1 |
| Warmup ratio | 0.05 |
| Weight decay | 0.01 |
| Max grad norm | 1.0 |
| Precision | AMP (mixed precision) |
| PPL upper bound | 10.0 (safety stop) |
| Metric | Value |
|---|---|
| Loss | ~0.01-0.10 (down from 0.693 initial) |
| Accuracy | ~87-100% |
| Perplexity | 4.70 (stable, initial: 4.68) |
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load the SFT-full base model
base_model = AutoModelForCausalLM.from_pretrained(
"Mishrakshitij/spar-qwen3-4b-sft-full",
torch_dtype="auto",
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Mishrakshitij/spar-qwen3-4b-sft-full")
# Apply DPO LoRA adapter
model = PeftModel.from_pretrained(base_model, "Mishrakshitij/spar-dpo-lora-adapter-ckpt1000")
# Optional: merge for faster inference
model = model.merge_and_unload()