Downloads · 30 days
304
35% of all-time downloads
str33t/madlad400-3b-mt-en-dv-news-v1-GGUF
madlad400-3b-mt-en-dv-news-v1-GGUF is a translation model from str33t. Use it when you need text moved from one language to another. The card lists the license as apache-2.0.
GGUF quantizations of str33t/madlad400-3b-mt-en-dv-news-v1, an English→Dhivehi (Thaana) translation model: MADLAD-400 3B-MT fine-tuned with LoRA on ~93k news-domain sentence pairs, with the adapter merged into the bas…
Downloads · 30 days
304
35% of all-time downloads
All-time downloads
872
Public
Repo size
12.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf12.2 GB · 100%
From the Hugging Face model README
GGUF quantizations of str33t/madlad400-3b-mt-en-dv-news-v1, an English→Dhivehi (Thaana) translation model: MADLAD-400 3B-MT fine-tuned with LoRA on ~93k news-domain sentence pairs, with the adapter merged into the base weights before conversion.
Recommended build: q4_k_m — 1.86 GB, no measurable quality loss, runs
on a plain CPU with no Python.
| File | Size | chrF++ | Δ vs bf16 | keymap round-trip |
|---|---|---|---|---|
| bf16 reference (not in this repo) | 5.88 GB | 60.88 | — | 100% |
| madlad3b-en-dv-q8_0.gguf | 3.13 GB | 60.88 | 0.00 | 100% |
| madlad3b-en-dv-q6_k.gguf | 2.42 GB | 61.02 | +0.14 | 100% |
| madlad3b-en-dv-q5_k_m.gguf | 2.13 GB | 63.10* | +0.11* | 100% |
| madlad3b-en-dv-q4_k_m.gguf | 1.86 GB | 60.91 | +0.03 | 100% |
| madlad3b-en-dv-q3_k_m.gguf | 1.47 GB | 60.09 | −0.79 | 99.9% |
| madlad3b-en-dv-q2_k.gguf | 1.18 GB | 57.30 | −3.58 | 100% |
* Q5_K_M was evaluated on a 150-sentence screen (baseline 62.99 on the same sentences); all other rows are the full 1,500 pairs.
With beam search (not available in llama.cpp for encoder-decoder models) the bf16 model scores 61.86 chrF++; greedy decoding costs ~1 chrF++ across the board, independent of quantization. Below 4 bits, degraded outputs are repetition loops of valid Thaana or occasional foreign-character substitutions, not broken script.
Full details, speed/memory measurements and per-tensor sensitivity analysis: see the paper in the project repository.
The model is a T5-family encoder-decoder. In llama.cpp use llama-completion
(the server does not support encoder-decoder models), greedy decoding, and
prefix the source text with the target-language token:
llama-completion -m madlad3b-en-dv-q4_k_m.gguf \
-p "<2dv> The government announced a new education policy yesterday." \
-n 256 --temp 0 -ngl 0 -no-cnv
<2dv> prefix is required (it selects Dhivehi as the target language).--temp 0 (greedy) is the evaluated setting.-ngl 0 = CPU only (~23 tok/s on 8 threads at q4_k_m, <3 GB RAM);
use -ngl 99 to offload to GPU (~178 tok/s).Base: google/madlad400-3b-mt
(Apache-2.0). LoRA fine-tune (r=32, attention+FFN, 3 epochs on ~93k pairs) as
described in the first paper;
adapter merged, saved with tie_word_embeddings=False (the MADLAD checkpoint
ships an untied lm_head that recent transformers versions mis-tie), converted
with llama.cpp convert_hf_to_gguf.py, quantized with llama-quantize.
Conversion fidelity check: 19/20 Q8_0 outputs byte-identical to the PyTorch
bf16 model on a parity slice.