asha-rajasthani-0.5b

Asha — Rajasthani Conversational Sales Assistant (0.5B)

A lightweight conversational model fine-tuned from Qwen2.5-0.5B-Instruct to act as a polite female virtual sales representative ("आशा") that replies in the same language or dialect the user speaks: Marwari (मारवाड़ी), Dhundhari (ढूंढाड़ी), Hindi, Hinglish, and English.

Why this base model

• Ungated, Apache-2.0, strong small-model multilingual/Devanagari coverage.
• Small enough to LoRA-fine-tune on CPU and to serve at ~200 ms first token on a laptop CPU as a Q4_K_M GGUF (~400 MB), i.e., near-zero VRAM requirement.

Training data

• Hand-built Marwari/Dhundhari/Hindi/Hinglish/English conversational seed set (greetings, small talk, sales-call dialogues: pricing, discounts, delivery, warranty, callbacks, objection handling) — the target-domain core.
• ai4bharat/indic-instruct-data-v0.1 (Hindi subsets) — general Hindi instruction following.
• Public Hinglish conversational/translation data — code-mixed fluency.

Method

LoRA (r=16, α=32, all attention + MLP projections), trained on CPU, merged and quantized to GGUF Q4_K_M for llama.cpp serving.

Evaluation

Scored 1–10 per response by a local judge model (Qwen2.5-1.5B-Instruct) combined with language/dialect-match and keyword heuristics over a multilingual question bank. See eval/scores.json for full history and per-language breakdowns.

Limitations (read this)

• 0.5B parameters: fluent conversational behavior, but factual knowledge is limited; it can and will make mistakes. No model achieves 0% error.
• Marwari and Dhundhari public text is extremely scarce; dialect quality comes from a small hand-built dataset and improves substantially with your own business data.
• Voice round-trip latency (STT + LLM + TTS) is seconds on CPU; the ~200 ms figure applies to the text first-token latency.

Serving

llama-server -m gguf/asha-0.5b-q4_k_m.gguf --port 8080 -t 6 -c 2048

Voice app (faster-whisper STT + edge-tts female voice): see app/ in the project repo.