indian-history-qlora

Indian History QLoRA Fine-tuned Model

This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct trained on NCERT Class XII Indian History textbook (128 pages) using QLoRA (4-bit quantization + LoRA adapters).

Model Details

Training Details

• Quantization: 4-bit NF4 with double quantization (bitsandbytes)
• LoRA rank: 16, alpha: 32
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Trainable parameters: 8.80M / 323.9M (2.72%)
• Optimizer: Paged AdamW 8-bit

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load adapter on top of base model
tokenizer = AutoTokenizer.from_pretrained("Rut-ai/indian-history-qlora")
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct",
    dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(model, "Rut-ai/indian-history-qlora")
model.eval()

# Generate
def ask(instruction, context=""):
    if context:
        prompt = f"### Instruction:\n{instruction}\n\n### Input:\n{context}\n\n### Response:\n"
    else:
        prompt = f"### Instruction:\n{instruction}\n\n### Response:\n"
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model.generate(**inputs, max_new_tokens=200, temperature=0.7,
                             do_sample=True, repetition_penalty=1.1)
    return tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

# Example
print(ask(
    "Continue the following passage about Indian history:",
    "The Mughal Empire was one of the largest empires in Indian history."
))

Training Data Format

The model was trained on text continuation in Alpaca format:

### Instruction:
Continue the following passage about Indian history:

### Input:
[First 150 words of a historical text chunk]

### Response:
[Continuation of the passage]

Limitations

• Small 0.5B model — may hallucinate specific dates/names on direct factual Q&A
• Best used for text continuation and passage completion tasks
• For factual Q&A, consider using RAG with the source PDF

License

Apache 2.0 — based on Qwen2.5-0.5B-Instruct