Downloads · 30 days
113
34% of all-time downloads
sulabhkatiyar/en-indic-translate-2b-GGUF
en-indic-translate-2b-GGUF is a text generation model from sulabhkatiyar. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as gemma.
GGUF builds of sulabhkatiyar/en-indic-translate-2b, for llama.cpp and the runtimes built on it (llama-server, llama-cli, LM Studio, Ollama-compatible loaders).
Downloads · 30 days
113
34% of all-time downloads
All-time downloads
333
Public
Repo size
12 GB
Likes
0
Public
Click a slice to open those files.
.gguf12 GB · 100%
From the Hugging Face model README
GGUF builds of sulabhkatiyar/en-indic-translate-2b,
for llama.cpp and the runtimes built on it (llama-server,
llama-cli, LM Studio, Ollama-compatible loaders).
The parent model translates English documents into 11 Indic languages — Assamese, Bengali,
Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil and Telugu — and is built for
whole documents rather than single sentences: long, densely structured, formula-heavy text where
LaTeX math, section headings and fenced code have to come through untouched. It is a fine-tune of
google/gemma-4-E2B-it.
Three quantizations are published here: Q8_0, Q5_K_M and Q4_K_M. Each one translated a real 99-document evaluation set end to end and was scored against the unquantized F16 build, so the numbers below are measured translation quality, not a proxy.
Start with Q5_K_M. On this model it is the best quality-per-byte of the three: on the
evaluation below it measures −0.07 chrF++ against F16 on the documents where both produced clean
output — i.e. no measurable quality cost — while being 2.565× smaller and decoding 1.485×
faster than F16.
| file | size | compression vs F16 | when to pick it |
|---|---|---|---|
en-indic-translate-2b-Q5_K_M.gguf | 3.381 GiB (3,630,269,920 B) | 2.565× | the default. No measurable quality cost against F16 (−0.07 chrF++ on 82 matched clean documents) and 1.485× decode — within 0.6 % of the fastest arm measured here. |
en-indic-translate-2b-Q8_0.gguf | 4.626 GiB (4,967,478,752 B) | 1.874× | the conservative choice. +0.11 chrF++ against F16 on 87 matched clean documents, 1.402× decode. The only arm in this sweep with a wrong-script failure (see below). |
en-indic-translate-2b-Q4_K_M.gguf | 3.192 GiB (3,427,861,984 B) | 2.716× | only if the last 0.19 GiB decides it. It costs −1.36 chrF++ against F16 on 76 matched clean documents and collapses into a repetition loop on 17 of 99 documents against F16's 9, and buys 0.6 % decode speed over Q5_K_M. |
F16 is not published in this repository. It appears in every table below as the reference
control — the unquantized build each quantization is measured against — and is not offered as a
download. For unquantized weights use the parent repo
sulabhkatiyar/en-indic-translate-2b.
sha256 of the published files, as built:
188d9dc92cf8633bb1c447ba2ecc48ca81dfe795fa4d5b0400eb9cce9b445d9c en-indic-translate-2b-Q8_0.gguf
ebfd9730d4030cb8fdab805177d78497f58ea2468ca3cf7c74edbab72a2185ee en-indic-translate-2b-Q5_K_M.gguf
8762dea9401e19e4eef88f9f60366665f527ffe6b2b193fbd1177c343c7384ae en-indic-translate-2b-Q4_K_M.gguf
Basis: 99 documents per arm — 11 languages × 9 documents, balanced. All four arms translate the
same 99 held-out English documents; chrF++ and spBLEU are corpus scores against reference
translations. (The generation run covered 100 documents, but one is missing from the Q4_K_M arm,
so every arm is scored on the 99 they all share.) F16 is the control, not a download.
| arm | chrF++ | spBLEU | Δ chrF++ vs F16, matched clean documents (n) | degenerate | target script ≥ 0.95 |
|---|---|---|---|---|---|
| F16 (reference control, not published here) | 73.59 | 63.41 | — | 9 / 99 | 98 / 99 |
| Q8_0 | 74.70 | 65.37 | +0.11 (n = 87) | 8 / 99 | 97 / 99 |
| Q5_K_M | 73.09 | 61.98 | −0.07 (n = 82) | 11 / 99 | 98 / 99 |
| Q4_K_M | 67.81 | 55.35 | −1.36 (n = 76) | 17 / 99 | 98 / 99 |
Reading the columns:
Q4_K_M's 17 degenerate documents
against 16 truncations is one document that looped and still finished cleanly with an
end-of-sequence token, which a truncation count cannot see. Apart from that one document, the
detector flags nothing that completed normally in any arm, so these counts are not padded by false
positives.(2003.02051, kn), sits at 0.915–0.917 in all four arms including F16 — it is a property of that
document (LaTeX and numerals), not of quantization. Q8_0's second miss is a genuine failure and is
described below.Does the GGUF format itself cost anything? No. The same 2b weights served as bf16 through vLLM score 0.10 chrF++ above the F16 GGUF on 85 matched clean documents — noise. Everything below F16 in the table is the quantization, not the file format.
Decode rates were measured on one NVIDIA Tesla T4 — the ratio to F16 is the transferable number, the absolute tok/s is not.
| arm | file size | compression | decode tok/s | vs F16 |
|---|---|---|---|---|
| F16 (reference control, not published here) | 8.672 GiB | 1.000× | 95.51 | 1.000× |
| Q8_0 | 4.626 GiB | 1.874× | 133.91 | 1.402× |
| Q5_K_M | 3.381 GiB | 2.565× | 141.81 | 1.485× |
| Q4_K_M | 3.192 GiB | 2.716× | 142.65 | 1.494× |
The speed-up saturates between Q5_K_M and Q4_K_M: the file is 5.6 % smaller and decodes
0.6 % faster, while the matched-clean delta against F16 goes from −0.07 (n = 82) to −1.36 (n = 76)
and six more documents collapse (11/99 to 17/99). There is no speed argument for Q4_K_M over
Q5_K_M on this hardware.
The smaller file also buys you room. Weights you no longer load are VRAM the KV cache can use, so on the same card you can keep more requests running at once, or give each one a longer context.
A separate, structural measurement: each quantization's output distribution compared with F16's on a
fixed corpus (CPU, 38 chunks, n_ctx 512). It reads the weights; the tables above read what the
model wrote.
| arm | Mean KLD | Same top p | PPL ratio |
|---|---|---|---|
| Q8_0 | 0.003366 | 97.307 % | 1.0279 |
| Q5_K_M | 0.053007 | 89.948 % | 1.0400 |
| Q4_K_M | 0.163456 | 82.993 % | 1.0897 |
Against the thresholds llama.cpp's own quantization scoreboard uses for each type, Q8_0 meets all
three; Q5_K_M misses the same-top-p bar by 0.052 of a percentage point (89.948 % against 90 %);
and Q4_K_M misses it by 2.0 points (82.993 % against 85 %). Generation says what those two misses
are worth: the near-miss costs nothing measurable (−0.07 chrF++ on 82 matched clean documents),
while the clear miss costs 1.36 chrF++ (on 76) and eight net collapsed documents.
KLD ordered the three arms in the same order as translation quality, but the step sizes do not track
the cost. The 15.7× KLD step from Q8_0 to Q5_K_M moves the matched-clean delta against F16
from +0.11 (n = 87) to −0.07 (n = 82) — nothing measurable. The smaller 3.1× step from Q5_K_M
to Q4_K_M moves it to −1.36 (n = 76) and adds six collapsed documents. The corpus is drawn from
private training data, so this is a distribution-shape comparison, not an absolute quality measure.
Two things were observed in the evaluation and are worth knowing before you deploy any of these files. Both are measured, not hypothetical.
In this sweep, Q8_0 — the highest-precision quantization here — translated one document,
(1811.10941, pa), into Hebrew/Yiddish script under a Punjabi request: target-script share
0.000353 over 14,168 letters, chrF++ 0.92 on that document against F16's 26.32. It ran to
exactly the same 14,579-token budget that F16 hit on the same document in correct Gurmukhi, so the
truncation count scores the two identically — one a correct-script overrun, the other a total
language failure.
A second, independent instance of the same failure was observed on the sibling 4b model, in the
lowest-precision arm (Q4_K_M), which produced a fluent Telugu translation under an Odia
label — that one terminated cleanly with EOS, used fewer tokens than F16, and was not degenerate,
so neither a truncation count nor a degeneracy detector could see it. KL divergence does not predict
it either: Q8_0 has the best KLD of any arm in this sweep and is the arm that failed.
Two instances, two models, opposite ends of the bit-width ladder. It is not ordered by precision. The only check that caught either one is a target-script check on the output, so if you run these models in production, run one: verify that the generated text is in the script you asked for, and regenerate or fall back if it is not.
Across the four arms of this sweep there were 44 truncations, and 44 of 44 were repetition
loops — the count of documents that ran out of budget without the repetition detector firing is
zero, in every arm. (The same holds on the 4b sibling: 20 truncations, 20 loops.) Raising
n_predict therefore buys more copies of the jammed text and fixes nothing; the fix for a loop is a
repetition penalty (the evaluation deliberately ran with none, so the counts above are the
no-penalty worst case).
Download one file:
pip install -U huggingface_hub
hf download sulabhkatiyar/en-indic-translate-2b-GGUF \
en-indic-translate-2b-Q5_K_M.gguf --local-dir .
The model was trained and evaluated on exactly one prompt, as a single user turn in the Gemma-4 chat template:
Translate English to {Language}:
{your English text}
{Language} is the English name of the target language: Assamese, Bengali, Gujarati, Hindi,
Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu. Use the chat template —
the evaluation fed the model token ids produced by the parent repo's
apply_chat_template(..., add_generation_prompt=True).
llama-cli, one documentllama-cli -m en-indic-translate-2b-Q5_K_M.gguf \
--jinja -st -ngl 99 -c 32768 -n 8192 \
--temp 0 --top-k 1 \
-p "Translate English to Hindi:
Gradient descent is the workhorse of modern machine learning. Given a loss function \$L(\theta)\$, the parameters are updated iteratively."
llama-serverllama-server -m en-indic-translate-2b-Q5_K_M.gguf \
--jinja -ngl 99 -c 32768 --port 8080
curl -s http://localhost:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"messages": [{"role": "user",
"content": "Translate English to Tamil:\nYour English document here."}],
"temperature": 0, "top_k": 1, "max_tokens": 8192
}'
Notes that matter in practice:
--temp 0 --top-k 1) is what every number on this card was measured with.3 × prompt_tokens + 2048 output tokens, capped at 30,720, inside a 49,152-token sequence.--chat-template-file chat_template.jinja, downloaded from
sulabhkatiyar/en-indic-translate-2b.convert_hf_to_gguf.py --outtype f16 from
sulabhkatiyar/en-indic-translate-2b,
then llama-quantize to each target type. No importance matrix (imatrix) was used and no
per-tensor overrides were applied — these are llama.cpp's default K-quant mixes.3057bb66c86c46d5781e50e85462a760ba7d1feb (2026-09-12, tag b10931),
built from source. Conversion, quantization, generation and the KL measurement all used this one
tree.--temp 0 --top-k 1, seed 42), llama-server with -np 8, flash-attention
on, f16 KV cache, every sampler in the default chain explicitly neutralised, no repetition
penalty, prompt caching off. One Tesla T4.The 4b sibling repo reaches a different conclusion, and it does not transfer.
sulabhkatiyar/en-indic-translate-4b-GGUF
recommends a different quantization, measured on a different model in a separate sweep. Nothing on
that card sets the default here, and nothing here sets the default there — each model was swept on
its own and each card should be read on its own numbers. The two cards' figures also stand on
different document sets (99 documents for 2b, 100 for 4b), so a 2b number and a 4b number must
never be placed in the same row.
| Code | Language | Code | Language |
|---|---|---|---|
| as | Assamese | mr | Marathi |
| bn | Bengali | or | Odia |
| gu | Gujarati | pa | Punjabi |
| hi | Hindi | ta | Tamil |
| kn | Kannada | te | Telugu |
| ml | Malayalam |
Released under the Gemma Terms of Use, inherited from
google/gemma-4-E2B-it through
sulabhkatiyar/en-indic-translate-2b.
Quantized by sulabhkatiyar.