qwen1.5b-hinglish-sft-v2

Qwen2.5-1.5B — Hinglish SFT (LoRA adapter)

A LoRA adapter that fine-tunes Qwen2.5-1.5B-Base into a Hindi–English (Hinglish) code-switched assistant. The base model is a document completer; this adapter teaches it to follow instructions and reply in natural romanized Hinglish (with English and some Devanagari Hindi support).

This is the SFT (v2) stage of an SFT → DPO project; the DPO stage (now available) addresses some of the limitations below.

Next stage: This SFT model is the reference for a DPO-aligned version that is preferred 68.2% of the time over this one (held-out, blind judge) with no capability loss — see qwen1.5b-hinglish-dpo and the writeup Part 2: Preference Optimization with DPO.

Model Details

Model Description

• Developed by: Sarthak Senapati
• Model type: Decoder-only causal LM (Qwen2.5) + LoRA adapter (PEFT), QLoRA-trained
• Language(s) (NLP): Romanized Hindi–English (Hinglish, primary), English, Hindi (Devanagari)
• License: Apache-2.0 (adapter); base model Qwen2.5-1.5B is Apache-2.0
• Finetuned from model: Qwen/Qwen2.5-1.5B

Model Sources

• Repository: https://github.com/sarthaksenapati/khichdi-hinglish-llm
• Dataset: https://huggingface.co/datasets/sarthaksenapati/khichdi-sft
• Blog / writeup (Part 1 — SFT): https://sarthak-senapati.hashnode.dev/teaching-a-base-model-to-speak-hinglish-part-1-sft
• DPO-aligned version: https://huggingface.co/sarthaksenapati/qwen1.5b-hinglish-dpo
• Writeup (Part 2 — DPO): https://sarthak-senapati.hashnode.dev/teaching-a-base-model-to-speak-hinglish-part-2-preference-optimization-with-dpo-from-scratch

Uses

Direct Use

A lightweight assistant for everyday, low-stakes tasks in Hinglish: recipes, simple how-tos, travel/tech/daily-life questions, short writing help. Best for romanized Hinglish input.

Downstream Use

A starting checkpoint for further alignment (DPO/IPO/KTO) on Hinglish preference data, or for task-specific fine-tuning on Indic code-switched applications.

Out-of-Scope Use

Not safety-aligned and not suitable for high-stakes use — medical, legal, or financial advice; factual lookups requiring reliability; or any application where confident misinformation is harmful. Do not deploy without a moderation/guardrail layer.

Bias, Risks, and Limitations

• Stopping / verbosity: the model often does not emit a clean stop token and can ramble past the answer into irrelevant text. Use the generation settings below (explicit eos_token_id, repetition_penalty, no_repeat_ngram_size) and a max_new_tokens cap. This was a target of the DPO stage, which reduced but did not fully eliminate verbosity (see the DPO model).
• Devanagari quality: Devanagari (8% of training) is markedly weaker than romanized Hinglish and can produce broken output. Romanized is the supported path.
• Sycophancy / factuality: training responses were generated by a larger LLM; the model can agree with and repeat dubious claims (e.g. pseudoscientific health advice) and make factual or arithmetic errors.
• Synthetic-data bias: most training data is synthetic, so it inherits the generator's style and blind spots.

Recommendations

Use only for casual, low-stakes Hinglish interaction; keep a human in the loop; apply the recommended decoding settings; treat any factual/health/financial content as unverified.

How to Get Started with the Model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

BASE = "Qwen/Qwen2.5-1.5B"
ADAPTER = "sarthaksenapati/qwen1.5b-hinglish-sft-v2"

# the model was trained with a plain ChatML template (no system prompt)
CHATML = ("{% for m in messages %}"
          "{{ '<|im_start|>' + m['role'] + '\n' + m['content'] + '<|im_end|>' + '\n' }}"
          "{% endfor %}"
          "{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}")

tok = AutoTokenizer.from_pretrained(ADAPTER)
tok.chat_template = CHATML
im_end = tok.convert_tokens_to_ids("<|im_end|>")

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb,
                                            device_map="auto", dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, ADAPTER).eval()

msgs = [{"role": "user", "content": "Yaar mera laptop bahut slow hai, kuch quick fixes batao."}]
text = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False)
ids = tok(text, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**ids, max_new_tokens=200, do_sample=False,
                     repetition_penalty=1.2, no_repeat_ngram_size=3,
                     eos_token_id=im_end, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))

Training Details

Training Data

10,000 instruction–response pairs (sarthaksenapati/khichdi-sft), ~80% romanized Hinglish / 12% English / 8% Devanagari Hindi. Built via a two-stage synthetic pipeline (Self-Instruct style, Llama-3.3-70B), then heuristic cleaning, language-ID relabeling, LLM-as-judge quality scoring (1–10 rubric), MinHash near-deduplication, and a disjoint train / preference / eval split.

Training Procedure

QLoRA: frozen 4-bit (NF4, double-quant) base + bf16 LoRA adapters on all seven linear projections. Chat formatting and loss masking were done manually (prompt tokens set to -100; loss on the assistant turn including <|im_end|>), with a plain transformers Trainer.

Preprocessing

ChatML formatting; loss computed only on assistant tokens; <|im_end|> force-appended on any sequence exceeding the max length so every example ends with the stop token.

Training Hyperparameters

• Training regime: bf16 mixed precision (QLoRA, NF4 4-bit base)
• LoRA: r=32, alpha=64, dropout=0.05, target = q/k/v/o/gate/up/down proj
• Optimizer / schedule: paged_adamw_8bit, cosine, lr=2e-4, warmup 30 steps
• Epochs: 2 · Max length: 2048
• Batch: per-device 4 × grad-accum 4 (effective 16)
• Gradient checkpointing: on (use_reentrant=False)

Speeds, Sizes, Times

~43 min on one RTX 4090 (24 GB). Trainable params 36.9M / 1.58B (≈2.34%). Adapter ≈148 MB.

Evaluation

Testing Data, Factors & Metrics

Testing Data

A 5% held-out validation split of the training set for eval_loss, plus a disjoint set of 300 held-out eval prompts (never seen in training) used for the win-rate and MMLU evaluation.

Metrics

• Validation cross-entropy (eval_loss).
• Win-rate vs base — blind pairwise preference judged by an independent model (DeepSeek), each pair scored in both orderings (a win counts only if it survives the position swap), with a Wilson 95% confidence interval and a mean-length check for length bias.
• MMLU (1,140-question slice, 0-shot) — capability-regression check.

Results

• Validation eval_loss: 1.187 (v2; v1 with rank 16 reached 1.233).
• Win-rate vs base: 97.9% of decisive pairs (95% CI 95.2–99.1%) on 300 held-out prompts, no length bias.
• MMLU: base 0.626 → SFT 0.618 — within noise; no measurable capability regression from SFT.
• Qualitatively, base vs SFT is a clear behavior change: the base model produces English document-completion / degenerate loops on the chat format, while the SFT model answers in Hinglish and follows instructions.

Summary

SFT converts the base model into a Hinglish instruction-follower that is overwhelmingly preferred over base with no capability tax. Remaining issues (clean stopping, verbosity, Devanagari quality) are documented above and were addressed in the DPO stage (now available).

Environmental Impact

• Hardware Type: NVIDIA RTX 4090 (24 GB)
• Hours used: ~0.7 (this run)
• Cloud Provider: RunPod
• Compute Region: varies
• Carbon Emitted: negligible (<~0.2 kg CO2eq, estimated)

Technical Specifications

Model Architecture and Objective

LoRA adapter over the decoder-only Qwen2.5-1.5B transformer. Objective: causal-LM cross-entropy on assistant tokens only (prompt masked).

Compute Infrastructure

Hardware

Single RTX 4090 (RunPod).

Software

transformers 5.x, peft 0.19.1, bitsandbytes, accelerate, datasets, Weights & Biases.

Citation

If you use this, please cite the repository and blog:

@misc{khichdi2026,
  title  = {Khichdi: post-training Qwen2.5-1.5B into a Hinglish assistant (SFT -> DPO)},
  author = {Senapati, Sarthak},
  year   = {2026},
  url    = {https://github.com/sarthaksenapati/khichdi-hinglish-llm}
}

Model Card Authors

Sarthak Senapati

Model Card Contact

https://github.com/sarthaksenapati

Framework versions

• PEFT 0.19.1