Downloads · 30 days
41
55% of all-time downloads
TigreGotico/nllb_200_distilled_600m-eng-amh-onnx
nllb_200_distilled_600m-eng-amh-onnx is a translation model from TigreGotico. Use it when you need text moved from one language to another. The card lists the license as apache-2.0.
ONNX export of dsfsi/nllb200distilled600m-eng-amh.
Downloads · 30 days
41
55% of all-time downloads
All-time downloads
75
Public
Repo size
15.1 GB
Likes
0
Public
Click a slice to open those files.
.onnx_data7.6 GB · 81%
From the Hugging Face model README
ONNX export of dsfsi/nllb_200_distilled_600m-eng-amh.
Translation direction: English (en) -> Amharic (am).
The original model was trained by DSFSI (Data
Science for Social Impact research group, University of Pretoria), a
fine-tune of facebook/nllb-200-distilled-600M on the afriscience-mt
dataset. Licence: Apache-2.0, as declared by the original repository.
This repository only converts the weights to ONNX. All credit for the
model belongs to DSFSI.
optimum-cli export onnx --model dsfsi/nllb_200_distilled_600m-eng-amh \
--task text2text-generation-with-past --no-post-process <outdir>
The int8 graphs in int8/ come from the fp32 graphs, one graph at a time,
with ONNX Runtime dynamic quantization:
from onnxruntime.quantization import QuantType, quantize_dynamic
quantize_dynamic(
"encoder_model.onnx",
"int8/encoder_model.onnx",
weight_type=QuantType.QInt8,
per_channel=True,
reduce_range=False,
use_external_data_format=True,
extra_options={"EnableSubgraph": True},
)
Each int8 graph keeps its weights in one external data file,
int8/<graph>.onnx_data. The script is
scripts/export/quantize_seq2seq_int8.py in
TigreGotico/linguonnx.
| Path | Precision | Size |
|---|---|---|
*.onnx + *.onnx_data (root) | fp32 | ~7 GB |
int8/*.onnx + int8/*.onnx_data | int8 dynamic, per channel | 1.9 GB |
10 general-domain English sentences, PyTorch original vs ONNX, exact string
match against AutoModelForSeq2SeqLM.generate().
parity:
sample_size: 10
metric: exact_match
fp32_greedy: 1.00 # 10/10
fp32_beam4: 1.00 # 10/10
| Decoding | fp32 (n=10) |
|---|---|
| greedy | 100% (10/10) |
| beam=4 | 100% (10/10) |
fp32 is an exact reproduction of the original model.
Caveat on sample size: n=10 exact-match is a small, noisy estimate, not a FLORES-scale benchmark. A separate measurement campaign on this project found n=5 exact-match numbers for another model swing from 100% to 14% when re-measured at n=100 on FLORES-200. Treat the percentages above as an initial signal only - a proper FLORES + chrF evaluation supersedes it wherever available.
The first 50 FLORES-200 devtest sentences, eng_Latn to amh_Ethi,
greedy decoding through linguonnx on ONNX Runtime 1.31.1. chrF is the
sacrebleu corpus chrF against the FLORES-200 reference.
| Build | Size | chrF | Outputs equal to fp32 |
|---|---|---|---|
| fp32 | 7.4 GB | 37.28 | 50/50 |
| int8 | 1.9 GB | 36.94 | 22/50 |
This is an NLLB checkpoint, not M2M100: the target language is not
baked into forced_bos_token_id in generation_config.json - you must
pass it explicitly at generate time using NLLB's flores_code language
tags (eng_Latn, amh_Ethi, not plain ISO 639-1 codes):
tokenizer.src_lang = "eng_Latn"
forced_bos_token_id = tokenizer.convert_tokens_to_ids("amh_Ethi")
model.generate(**inputs, forced_bos_token_id=forced_bos_token_id, ...)
from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM
repo = "TigreGotico/nllb_200_distilled_600m-eng-amh-onnx"
tok = AutoTokenizer.from_pretrained(repo)
tok.src_lang = "eng_Latn"
model = ORTModelForSeq2SeqLM.from_pretrained(repo, use_cache=True, use_merged=False)
enc = tok("The weather is very nice today.", return_tensors="pt")
tgt_id = tok.convert_tokens_to_ids("amh_Ethi")
out = model.generate(**enc, forced_bos_token_id=tgt_id, num_beams=4, max_new_tokens=64)
print(tok.batch_decode(out, skip_special_tokens=True)[0])
# -> ዛሬ የአየር ሁኔታ በጣም ጥሩ ነው፡፡
For the int8 build, pass subfolder="int8".