Downloads · 30 days
21
26% of all-time downloads
sarthaksenapati/qwen1.5b-hinglish-ipo
qwen1.5b-hinglish-ipo is a text generation model from sarthaksenapati. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
An IPO (Identity Preference Optimization)-aligned LoRA adapter for the Hindi–English (romanized Hinglish) assistant. This is one of three preference-optimization variants (DPO / IPO / KTO) trained on the same data to…
Downloads · 30 days
21
26% of all-time downloads
All-time downloads
82
Public
Repo size
85.3 MB
Likes
0
Public
Click a slice to open those files.
.safetensors73.9 MB · 87%
From the Hugging Face model README
An IPO (Identity Preference Optimization)-aligned LoRA adapter for the Hindi–English (romanized Hinglish) assistant. This is one of three preference-optimization variants (DPO / IPO / KTO) trained on the same data to compare methods; it is published mainly as part of that comparison.
IPO replaces DPO's log-sigmoid loss with a squared loss that regresses the policy–reference margin toward a finite target 1/(2β), rather than pushing it to infinity. This makes it more conservative and resistant to overfitting deterministic preferences. The loss was implemented from scratch (src/dpo/loss.py, ipo_loss) and verified against its reference formula and analytic invariants (loss = (1/(2β))² = 25.0 at init for β=0.1).
The IPO LoRA was trained on top of the merged SFT model, which is therefore its reference policy.
sarthaksenapati/qwen1.5b-hinglish-sft-v2 (from Qwen/Qwen2.5-1.5B)Low-stakes conversational assistance in Hinglish. In practice, prefer the KTO or DPO sibling — IPO did not significantly outperform the SFT baseline in this setup (see Results).
Not safety-aligned. No high-stakes use (medical/legal/financial), no reliance on factual correctness.
Same family of limitations as the SFT model: weak stop-control (rambling / no clean stop), weak Devanagari, occasional fabricated entities and confident unverified claims, and synthetic-data style bias. See reports/error_analysis.md in the repo. Additionally: with the shared hyperparameters used here, IPO under-performed — its squared loss is more sensitive to β than DPO's, and the DPO-tuned β=0.1 likely under-drove it.
This adapter was trained on top of the merged SFT model — apply the SFT adapter and merge it first, then load this IPO adapter:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
BASE = "Qwen/Qwen2.5-1.5B"
SFT = "sarthaksenapati/qwen1.5b-hinglish-sft-v2"
IPO = "sarthaksenapati/qwen1.5b-hinglish-ipo"
CHATML = ("{% for m in messages %}"
"{{ '<|im_start|>' + m['role'] + '\n' + m['content'] + '<|im_end|>' + '\n' }}"
"{% endfor %}"
"{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}")
tok = AutoTokenizer.from_pretrained(IPO); tok.chat_template = CHATML
im_end = tok.convert_tokens_to_ids("<|im_end|>")
base = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")
base = PeftModel.from_pretrained(base, SFT).merge_and_unload() # merge SFT (the reference)
model = PeftModel.from_pretrained(base, IPO).eval() # apply IPO adapter
msgs = [{"role": "user", "content": "Ek achhi morning routine ke liye tips do."}]
text = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False)
ids = tok(text, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**ids, max_new_tokens=256, do_sample=False,
repetition_penalty=1.2, no_repeat_ngram_size=3, eos_token_id=im_end)
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))
sarthaksenapati/khichdi-pref — 1,497 on-policy preference pairs (same as DPO).disable_adapter() recovers the reference.(1/(2β))² (correctness probe).khichdi-sft (eval_prompts), unseen in training.IPO preserved capability and answer length but, with hyperparameters shared across all three methods, did not separate from the SFT baseline. The likely cause is that IPO's squared loss needs its own β tuning. For practical use, prefer the DPO or KTO sibling. Published for transparency and as part of the DPO/IPO/KTO comparison (see the repo and the write-up).
Qwen2.5-1.5B decoder-only transformer + LoRA; objective = IPO squared loss with the SFT model as frozen reference. Software: PyTorch, transformers, PEFT, bf16; DeepSeek (via OpenRouter) for evaluation.
@misc{khichdi2026,
title = {Khichdi: post-training Qwen2.5-1.5B into a Hinglish assistant (SFT -> DPO/IPO/KTO)},
author = {Senapati, Sarthak},
year = {2026},
url = {https://github.com/sarthaksenapati/khichdi-hinglish-llm}
}
Sarthak Senapati