northwind-hr-policy-dpo-merged

Northwind HR Policy Assistant — DPO-Aligned

A domain-specific assistant for HR Policy, fine-tuned from unsloth/Qwen2.5-1.5B using Unsloth with LoRA/QLoRA. Format of this repo: merged.

Training pipeline

1. Non-instruction fine-tuning — domain adaptation on raw hr policy text.
2. Instruction fine-tuning (SFT) — supervised tuning on instruction→response pairs.
3. DPO alignment — preference tuning for safer, more professional answers.

Hyperparameters

• LoRA rank / alpha / dropout: 16 / 16 / 0.0
• Quantization: 4-bit QLoRA
• SFT learning rate: 0.0002; DPO learning rate: 5e-06; DPO beta: 0.1
• Effective batch size: 8
• Max sequence length: 1024

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer


repo = "meghaGenAI/northwind-hr-policy-dpo-merged"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

msgs = [{"role": "system", "content": "You are the Northwind HR Policy Assistant. Answer user questions about hr policy clearly, accurately, and professionally. If something is outside your scope, say so and point the user to the right team."},
        {"role": "user", "content": "Your question here"}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(inputs, max_new_tokens=200)[0]))

System prompt

You are the Northwind HR Policy Assistant. Answer user questions about hr policy clearly, accurately, and professionally. If something is outside your scope, say so and point the user to the right team.

Built with Unsloth. This model is a domain-specific assistant; verify critical answers against authoritative sources.