Downloads · 30 days
73
22% of all-time downloads
sulabhkatiyar/en-indic-translate-4b-GGUF
en-indic-translate-4b-GGUF is a text generation model from sulabhkatiyar. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as gemma.
GGUF builds of sulabhkatiyar/en-indic-translate-4b, for llama.cpp and the runtimes built on it (llama-server, llama-cli, LM Studio, Ollama-compatible loaders).
Downloads · 30 days
73
22% of all-time downloads
All-time downloads
327
Public
Repo size
19.1 GB
Likes
0
Public
Click a slice to open those files.
.gguf19.1 GB · 100%
From the Hugging Face model README
GGUF builds of sulabhkatiyar/en-indic-translate-4b,
for llama.cpp and the runtimes built on it (llama-server,
llama-cli, LM Studio, Ollama-compatible loaders).
The parent model translates English documents into 11 Indic languages — Assamese, Bengali,
Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil and Telugu — and is built for
whole documents rather than single sentences: long, densely structured, formula-heavy text where
LaTeX math, section headings and fenced code have to come through untouched. It is a fine-tune of
google/gemma-4-E4B-it.
Three quantizations are published here: Q8_0, Q5_K_M and Q4_K_M. Each one translated a real 100-document evaluation set end to end and was scored against the unquantized F16 build, so the numbers below are measured translation quality, not a proxy.
Start with Q8_0. It is the only quantization of this model that the evaluation can call
equivalent to F16: −0.01 chrF++ on the documents where both produced clean output, not one
document worse by 2 chrF++, and it is the only arm in the sweep that degenerated on nothing
(0 of 100) — it even fixed both of the loops F16 itself fell into. It is 1.874× smaller than
F16.
| file | size | compression vs F16 | when to pick it |
|---|---|---|---|
en-indic-translate-4b-Q8_0.gguf | 7.480 GiB (8,031,224,320 B) | 1.874× | the default. −0.01 chrF++ against F16 on 98 matched clean documents; 0/100 degenerate; no script failure. |
en-indic-translate-4b-Q5_K_M.gguf | 5.367 GiB (5,762,896,384 B) | 2.612× | the size choice, and it has a real cost: −0.92 chrF++ against F16 on 95 matched clean documents, spread across 9 of the 11 languages. It buys 2.113 GiB over Q8_0. Cleanest script record of any arm (100/100). |
en-indic-translate-4b-Q4_K_M.gguf | 4.969 GiB (5,335,273,984 B) | 2.821× | only if the last 0.398 GiB decides it. −3.03 chrF++ against F16 on 84 matched clean documents, 15 of 100 documents collapsed into repetition loops against F16's 2, and it is the arm that produced a whole document in the wrong language (see below). |
F16 is not published in this repository. It appears in every table below as the reference
control — the unquantized build each quantization is measured against — and is not offered as a
download. For unquantized weights use the parent repo
sulabhkatiyar/en-indic-translate-4b.
sha256 of the published files, as built:
fb317572cae888e09659046b9b028de055ae9d47f873d2031dcceb7958f7a387 en-indic-translate-4b-Q8_0.gguf
136a68096c598ec0925b36d3f09e8f0c174f6c7ab9e5ff2fc98ad03f79391295 en-indic-translate-4b-Q5_K_M.gguf
d1be705ef5b69f8311628ec999394031a25dd5a0c8a7e3ec2979dfb444730698 en-indic-translate-4b-Q4_K_M.gguf
Basis: 100 documents per arm — 11 languages, as with 10 documents and every other language 9
(unbalanced by design). All four arms translate the same 100 held-out English documents; chrF++
and spBLEU are corpus scores against reference translations. F16 is the control, not a download.
| arm | chrF++ | spBLEU | Δ chrF++ vs F16, matched clean documents (n) | degenerate | target script ≥ 0.95 |
|---|---|---|---|---|---|
| F16 (reference control, not published here) | 82.22 | 78.59 | — | 2 / 100 | 99 / 100 |
| Q8_0 | 82.67 | 79.41 | −0.01 (n = 98) | 0 / 100 | 99 / 100 |
| Q5_K_M | 80.15 | 75.43 | −0.92 (n = 95) | 3 / 100 | 100 / 100 |
| Q4_K_M | 70.52 | 60.20 | −3.03 (n = 84) | 15 / 100 | 99 / 100 |
Reading the columns:
Q8_0's single miss is the same document, (2003.02051, kn), at ~0.916 with correct Kannada as
its dominant script — a property of that document (LaTeX and numerals), not of quantization.
Q4_K_M's miss is a genuine failure and is described below.Where each arm's loss actually sits. Q8_0's −0.01 is noise: not a single document moves by 2
chrF++ and its worst is −1.81. Q5_K_M's −0.92 is systematic rather than one unlucky document — the
median document loses 0.79, 14 documents lose more than 2, and the loss is present in 9 of the 11
languages. Q4_K_M's −3.03 is broader still: half of its matched-clean documents lose more than 2
chrF++, and removing the wrong-language document below still leaves −2.28 on the remaining 83.
Does the GGUF format itself cost anything? No. The same 4b weights served as bf16 through vLLM score 0.06 chrF++ below the F16 GGUF on 96 matched clean documents — noise. Everything below F16 in the table is the quantization, not the file format.
| arm | file size | compression vs F16 |
|---|---|---|
| F16 (reference control, not published here) | 14.019 GiB | 1.000× |
| Q8_0 | 7.480 GiB | 1.874× |
| Q5_K_M | 5.367 GiB | 2.612× |
| Q4_K_M | 4.969 GiB | 2.821× |
These files decode up to 1.35× faster than F16 on the same GPU.
The smaller file also buys you room. Weights you no longer load are VRAM the KV cache can use, so on the same card you can keep more requests running at once, or give each one a longer context.
A separate, structural measurement: each quantization's output distribution compared with F16's on a
fixed corpus (CPU, 38 chunks, n_ctx 512). It reads the weights; the tables above read what the
model wrote.
| arm | Mean KLD | Same top p | PPL ratio |
|---|---|---|---|
| Q8_0 | 0.004156 | 97.368 % | 1.0297 |
| Q5_K_M | 0.067915 | 90.175 % | 1.0802 |
| Q4_K_M | 0.214926 | 81.858 % | 1.2047 |
Against the thresholds llama.cpp's own quantization scoreboard uses for each type, Q8_0 meets all
three; Q5_K_M misses on PPL ratio (1.0802 against a 1.06 bar) while passing the other two; and
Q4_K_M misses on both same-top-p (81.858 % against 85 %) and PPL ratio (1.2047 against 1.12).
Generation agrees with the ordering but not with the step sizes: Q8_0 → Q5_K_M is a 16.3×
KLD step that moves the matched-clean delta against F16 from −0.01 (n = 98) to −0.92 (n = 95), while
the smaller 3.2× step to Q4_K_M moves it to −3.03 (n = 84) and adds 12 more collapsed
documents (3/100 to 15/100). The corpus is drawn from private training data, so this is a
distribution-shape comparison, not an absolute quality measure.
Two things were observed in the evaluation and are worth knowing before you deploy any of these files. Both are measured, not hypothetical.
In this sweep, Q4_K_M translated one document, (1811.10740, or), into fluent, well-formed
Telugu under an Odia request: target-script ratio 0.000000 over 5,438 letters, chrF++
18.43 on that document against F16's 81.24. It terminated cleanly with EOS, used fewer
tokens than F16 and was not degenerate — so a truncation count, a degeneracy detector and a token-
excess check are all blind to it.
A second, independent instance was observed on the sibling 2b model, in the highest-precision arm
(Q8_0), which produced a document in Hebrew/Yiddish script under a Punjabi request. KL
divergence does not predict this failure either: in that 2b sweep the failing arm had the best KLD of
any arm measured.
Two instances, two models, opposite ends of the bit-width ladder. It is not ordered by precision. The only check that caught either one is a target-script check on the output, so if you run these models in production, run one: verify that the generated text is in the script you asked for, and regenerate or fall back if it is not.
Across the four arms of this sweep there were 20 truncations, and 20 of 20 were repetition
loops — the count of documents that ran out of budget without the repetition detector firing is
zero, in every arm. (The same holds on the 2b sibling: 44 truncations, 44 loops.) Raising
n_predict therefore buys more copies of the jammed text and fixes nothing; the fix for a loop is a repetition
penalty (the evaluation deliberately ran with none, so the counts above are the no-penalty worst
case). The token cost is concentrated in exactly those documents: Q4_K_M generated 15.7 % more
tokens than F16 overall, and on the 84 documents where both stayed healthy it generated slightly
fewer — it is not a more verbose model, it loops on 15 documents.
Download one file:
pip install -U huggingface_hub
hf download sulabhkatiyar/en-indic-translate-4b-GGUF \
en-indic-translate-4b-Q8_0.gguf --local-dir .
The model was trained and evaluated on exactly one prompt, as a single user turn in the Gemma-4 chat template:
Translate English to {Language}:
{your English text}
{Language} is the English name of the target language: Assamese, Bengali, Gujarati, Hindi,
Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu. Use the chat template —
the evaluation fed the model token ids produced by the parent repo's
apply_chat_template(..., add_generation_prompt=True).
llama-cli, one documentllama-cli -m en-indic-translate-4b-Q8_0.gguf \
--jinja -st -ngl 99 -c 32768 -n 8192 \
--temp 0 --top-k 1 \
-p "Translate English to Hindi:
Gradient descent is the workhorse of modern machine learning. Given a loss function \$L(\theta)\$, the parameters are updated iteratively."
llama-serverllama-server -m en-indic-translate-4b-Q8_0.gguf \
--jinja -ngl 99 -c 32768 --port 8080
curl -s http://localhost:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"messages": [{"role": "user",
"content": "Translate English to Tamil:\nYour English document here."}],
"temperature": 0, "top_k": 1, "max_tokens": 8192
}'
Notes that matter in practice:
--temp 0 --top-k 1) is what every number on this card was measured with.3 × prompt_tokens + 2048 output tokens, capped at 30,720, inside a 49,152-token sequence.-ngl 99 offloads every
layer it can; lower it (or lower -c) if you run out of VRAM.--chat-template-file chat_template.jinja, downloaded from
sulabhkatiyar/en-indic-translate-4b.convert_hf_to_gguf.py --outtype f16 from
sulabhkatiyar/en-indic-translate-4b,
then llama-quantize to each target type. No importance matrix (imatrix) was used and no
per-tensor overrides were applied — these are llama.cpp's default K-quant mixes.3057bb66c86c46d5781e50e85462a760ba7d1feb (2026-09-12, tag b10931),
built from source. Conversion, quantization, generation and the KL measurement all used this one
tree.as with 10 and every other
language 9 — drawn from a 500-document held-out set that is disjoint from the parent model's
training data by document id. Every arm produced every document; scores are chrF++ and spBLEU
against reference translations for the same documents.--temp 0 --top-k 1, seed 42), llama-server with -np 8,
flash-attention on, f16 KV cache, every sampler in the default chain explicitly neutralised, no
repetition penalty, prompt caching off. Kaggle Tesla T4.The 2b sibling repo reaches a different conclusion, and it does not transfer.
sulabhkatiyar/en-indic-translate-2b-GGUF
recommends a different quantization, measured on a different model in a separate sweep. Nothing on
that card sets the default here, and nothing here sets the default there — each model was swept on
its own and each card should be read on its own numbers. The two cards' figures also stand on
different document sets (99 documents for 2b, 100 for 4b), so a 2b number and a 4b number must
never be placed in the same row.
| Code | Language | Code | Language |
|---|---|---|---|
| as | Assamese | mr | Marathi |
| bn | Bengali | or | Odia |
| gu | Gujarati | pa | Punjabi |
| hi | Hindi | ta | Tamil |
| kn | Kannada | te | Telugu |
| ml | Malayalam |
Released under the Gemma Terms of Use, inherited from
google/gemma-4-E4B-it through
sulabhkatiyar/en-indic-translate-4b.
Quantized by sulabhkatiyar.