gemma4-tamil-e4b-it-GGUF

Gemma 4 E4B — Tamil Instruction-Tuned (GGUF)

GGUF-quantized builds of prasadvittaldev/gemma4-tamil-e4b-it, for running locally with llama.cpp, Ollama, or LM Studio — no Python/GPU-inference stack required.

How this was built

1. Data collection — four existing public Tamil instruction/conversation datasets were identified, each covering a different gap: a broad multi-task instruction set (Aya_Tamil), conversational/safety-alignment data (indic-align), a general Alpaca/Orca-style instruction set (tamil-alpaca-orca), and a small natural Tamil-English code-switching sample (Chat2Find).
2. Normalization — each source was converted into one unified {"messages": [...]} chat format compatible with Gemma's chat template, with light per-source cleanup (stratified sampling where a source was too large, Tamil-majority filtering for the code-switching source).
3. Deduplication and merge — all four sources were combined and deduplicated on normalized text, producing one ~300K-row training set.
4. Fine-tuning — QLoRA (rank 16, alpha 32, targeting attention + MLP projections) via Unsloth, 2 epochs, on a single local RTX 5060 Ti (16GB).
5. Merge — the trained LoRA adapter was merged back into the base model to produce one standalone 16-bit checkpoint (prasadvittaldev/gemma4-tamil-e4b-it).
6. GGUF conversion — the merged checkpoint was converted to GGUF (bf16) and quantized down to Q5_K_M for practical local use in llama.cpp-based tools.
7. QA — manual sign-off against a small set of representative prompts (pure Tamil, Tanglish + numerals, a refusal check), run directly against the quantized GGUF via llama-cli, before release.

Files

A Q4_K_M build may be added in a future update.

Model details

• Base model: unsloth/gemma-4-E4B-it-unsloth-bnb-4bit (Gemma 4 E4B, instruction-tuned)
• Method: QLoRA, r=16, alpha=32, merged to 16-bit, then GGUF-converted and quantized
• Training data: prasadvittaldev/gemma4-tamil-sft — combined and deduplicated from Cognitive-Lab/Aya_Tamil, ai4bharat/indic-align, abhinand/tamil-alpaca-orca, and Chat2Find/Chat2Find-Instruct-Reasoning-Sample
• Epochs: 2 (final training loss: 0.031)

License

Released under CC-BY-NC-4.0 (non-commercial) — not the same as Gemma 4's own Apache 2.0 license. The more restrictive non-commercial terms apply here because part of the training mix (abhinand/tamil-alpaca-orca) carries a non-commercial license. Usable for research, evaluation, and personal projects; not for commercial deployment or resale.

Intended use

Tamil-language instruction following and conversation, with tolerance for natural Tamil-English code-switching ("Tanglish"). Designed as the reasoning/response-generation stage of a real-time Tamil voice assistant, packaged here for easy local use without a Python/CUDA stack.

Limitations

• Evaluated only via a small manual QA set (3 representative prompts: pure Tamil, Tanglish + numerals, a refusal check), not comprehensive benchmarking.
• Refusal/safety-decline responses were observed to be functionally correct (no harmful content generated) but occasionally grammatically rougher than the model's general fluency on non-refusal prompts.
• Not evaluated on code, math, or long-document reasoning.
• Gemma 4's chat template supports a "thinking" mode that some tools enable by default. If your tool's default settings produce an English planning/reasoning preamble before the Tamil answer (or cut off before reaching it), look for a "disable thinking/reasoning" toggle — in raw llama.cpp/llama-cli this is -rea off (or --reasoning off). With reasoning off, responses are direct and fully in Tamil, matching the QA results below.

How to use

Option 1 — Ollama

Step 1 — download a GGUF file from this repo's Files tab (e.g. gemma4-tamil-e4b-it-Q5_K_M.gguf).

Step 2 — create a Modelfile in the same directory. Setting num_predict gives generations room to complete; if you see an English planning/"thinking" preamble before the Tamil answer, check Ollama's reasoning/thinking settings for this model and disable it.

FROM ./gemma4-tamil-e4b-it-Q5_K_M.gguf
PARAMETER temperature 0.3
PARAMETER num_predict 512

Step 3 — build the Ollama model:

ollama create gemma4-tamil -f Modelfile

Step 4 — run it:

ollama run gemma4-tamil "தமிழ் மொழியின் வரலாறு பற்றி சுருக்கமாக சொல்லுங்கள்."

Option 2 — LM Studio

Step 1 — open LM Studio and go to the search/discover tab.

Step 2 — search for prasadvittaldev/gemma4-tamil-e4b-it-GGUF.

Step 3 — download your preferred quantization (Q5_K_M recommended).

Step 4 — load the model in LM Studio's chat interface and start chatting in Tamil (or Tanglish).

Option 3 — llama.cpp directly

Step 1 — download a GGUF file from this repo (or point llama-cli at the repo directly with -hf):

llama-cli -hf prasadvittaldev/gemma4-tamil-e4b-it-GGUF:Q5_K_M \
  -cnv -st -rea off --temp 0.3 -n 250 \
  -p "தமிழ் மொழியின் வரலாறு பற்றி சுருக்கமாக சொல்லுங்கள்."

Step 2 (alternative) — run against a local file if you downloaded it manually:

llama-cli -m gemma4-tamil-e4b-it-Q5_K_M.gguf \
  -cnv -st -rea off --temp 0.3 -n 250 \
  -p "தமிழ் மொழியின் வரலாறு பற்றி சுருக்கமாக சொல்லுங்கள்."

-rea off disables Gemma 4's chat-template "thinking" mode, which llama-cli otherwise enables by default — without it, the model produces an English planning preamble before (or instead of) the Tamil answer.

QA verification

The Q5_K_M GGUF was tested directly via llama-cli (-rea off, --temp 0.3) against the same 3 prompts used to sign off the source model, with results matching or exceeding the original: pure Tamil fluency (coherent historical prose), Tanglish + numeral handling (correctly parsed the code-switched query, replied fluently in Tamil), and a refusal check (clean, clearly-worded decline citing ethical/safety concerns, no harmful content).

Contact

For questions, feedback, or further discussion, reach out at [email protected].