Qwen3-0.6B-Hindi-Instruct-v1

⚠️ Note: This 0.6B version is undertrained and does not reliably follow Hindi instructions. For a working Hindi model, please use Qwen3-4B-Hindi-Instruct-v2 (GGUF here).

---

Qwen3-0.6B Hindi Instruct v1

A Qwen3-0.6B model fine-tuned on Hindi instruction data — built for developers and researchers who need a tiny, fast, Hindi-capable language model that runs anywhere.

This is the full precision safetensors version. For local CPU inference, use the GGUF version instead.

---

Quick Start

Load and run with transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "भारत की राजधानी क्या है?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Model Details

---

Training Details

This model was fine-tuned using QLoRA — a memory-efficient technique that trains small adapter weights on top of a frozen base model. This makes it possible to fine-tune on free Google Colab without any paid GPU.

Training setup:

Base model:   Qwen/Qwen3-0.6B
Method:       QLoRA via Unsloth
LoRA rank:    16
LoRA alpha:   16
Batch size:   2
Grad accum:   4
LR:           2e-4
Steps:        60
Hardware:     Google Colab free tier (T4 GPU)

---

Example Outputs

Hindi Question Answering:

User: भारत की राजधानी क्या है?
Model: भारत की राजधानी नई दिल्ली है।

Hindi Instructions:

User: मुझे चाय बनाने का तरीका बताओ।
Model: चाय बनाने के लिए पहले पानी गरम करें...

Mixed Language Coding:

User: Python में for loop कैसे लिखते हैं?
Model: Python में for loop इस तरह लिखते हैं...

---

Intended Use

• Hindi language applications and chatbots
• Low-resource Hindi NLP research
• Edge deployment where model size matters
• Fine-tuning base for domain-specific Hindi tasks
• Educational projects around Hindi AI

---

Limitations

• Trained on only 2000 examples — v2 will have significantly more data
• May mix Hindi and English in some responses
• Not trained for harmful content filtering — use responsibly
• Small model size means limited reasoning on complex tasks

---

Why This Model?

• Tiny — 600M parameters, runs on any machine
• Hindi-first — specifically fine-tuned for Hindi instruction following
• Open — Apache 2.0, free for personal and commercial use
• Reproducible — trained entirely on free Colab, full pipeline documented
• Versioned — actively maintained with improvements planned

---

Roadmap

• Done: v1 — Base Hindi fine-tune on Qwen3-0.6B with 2000 samples
• Next: v2 — 10x larger dataset, improved Hindi fluency
• Next: v3 — RLHF alignment for better instruction following
• Next: Qwen3-1.7B-Hindi — bigger model, same niche
• Next: Live demo Space on HuggingFace

---

Related Repos

---

Citation

If you use this model in your research or project, please cite:

@misc{pankajpandey-dev-hindi-2026,
  author    = {Pankaj Pandey},
  title     = {Qwen3-0.6B Hindi Instruct v1},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1}
}

---

About the Author

Made by pankajpandey-dev Building open-source Hindi AI models for India

Follow for weekly model updates and new Hindi LLM releases.

Found this useful? Please like this repo — it helps other Hindi speakers find it.