Downloads · 30 days
51
65% of all-time downloads
TigreGotico/nllb_200_distilled_600m-yor-eng-onnx
nllb_200_distilled_600m-yor-eng-onnx is a translation model from TigreGotico. Use it when you need text moved from one language to another. The card lists the license as apache-2.0.
ONNX export of dsfsi/nllb200distilled600m-yor-eng.
Downloads · 30 days
51
65% of all-time downloads
All-time downloads
79
Public
Repo size
15.1 GB
Likes
0
Public
Click a slice to open those files.
.onnx_data7.6 GB · 81%
From the Hugging Face model README
ONNX export of dsfsi/nllb_200_distilled_600m-yor-eng.
Translation direction: Yoruba (yo) -> English (en).
The original model was trained by DSFSI (Data
Science for Social Impact research group, University of Pretoria), a
fine-tune of facebook/nllb-200-distilled-600M on the afriscience-mt
dataset. Licence: Apache-2.0, as declared by the original repository.
This repository only converts the weights to ONNX. All credit for the
model belongs to DSFSI.
optimum-cli export onnx --model dsfsi/nllb_200_distilled_600m-yor-eng \
--task text2text-generation-with-past --no-post-process <outdir>
The int8 graphs in int8/ come from the fp32 graphs, one graph at a time,
with ONNX Runtime dynamic quantization:
from onnxruntime.quantization import QuantType, quantize_dynamic
quantize_dynamic(
"encoder_model.onnx",
"int8/encoder_model.onnx",
weight_type=QuantType.QInt8,
per_channel=True,
reduce_range=False,
use_external_data_format=True,
extra_options={"EnableSubgraph": True},
)
Each int8 graph keeps its weights in one external data file,
int8/<graph>.onnx_data. The script is
scripts/export/quantize_seq2seq_int8.py in
TigreGotico/linguonnx.
| Path | Precision | Size |
|---|---|---|
*.onnx + *.onnx_data (root) | fp32 | ~7 GB |
int8/*.onnx + int8/*.onnx_data | int8 dynamic, per channel | 1.9 GB |
10 general-domain Yoruba sentences (source language for this yo->en
pair), PyTorch original vs ONNX, exact string match against
AutoModelForSeq2SeqLM.generate().
parity:
sample_size: 10
metric: exact_match
fp32_greedy: 1.00 # 10/10
fp32_beam4: 1.00 # 10/10
| Decoding | fp32 (n=10) |
|---|---|
| greedy | 100% (10/10) |
| beam=4 | 100% (10/10) |
fp32 is an exact reproduction of the original model.
Caveat on sample size: n=10 exact-match is a small, noisy estimate, not a FLORES-scale benchmark. A separate measurement campaign on this project found n=5 exact-match numbers for another model swing from 100% to 14% when re-measured at n=100 on FLORES-200. Treat the percentages above as an initial signal only - a proper FLORES + chrF evaluation supersedes it wherever available.
The first 50 FLORES-200 devtest sentences, yor_Latn to eng_Latn,
greedy decoding through linguonnx on ONNX Runtime 1.31.1. chrF is the
sacrebleu corpus chrF against the FLORES-200 reference.
| Build | Size | chrF | Outputs equal to fp32 |
|---|---|---|---|
| fp32 | 7.4 GB | 44.92 | 50/50 |
| int8 | 1.9 GB | 45.69 | 13/50 |
On "Ìjọba kéde àwọn ìlànà ọrọ̀-ajé tuntun lánàá." (lánàá is
"yesterday"), fp32 gives "The government announced new economic policies
yesterday." and int8 gives "The government announced new economic policies
today.", with greedy decoding and with num_beams=4. Check time references
in int8 output from this model.
This is an NLLB checkpoint, not M2M100: the target language is not
baked into forced_bos_token_id in generation_config.json - you must
pass it explicitly at generate time using NLLB's flores_code language
tags (yor_Latn, eng_Latn, not plain ISO 639-1 codes):
tokenizer.src_lang = "yor_Latn"
forced_bos_token_id = tokenizer.convert_tokens_to_ids("eng_Latn")
model.generate(**inputs, forced_bos_token_id=forced_bos_token_id, ...)
from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM
repo = "TigreGotico/nllb_200_distilled_600m-yor-eng-onnx"
tok = AutoTokenizer.from_pretrained(repo)
tok.src_lang = "yor_Latn"
model = ORTModelForSeq2SeqLM.from_pretrained(repo, use_cache=True, use_merged=False)
enc = tok("Ojú-ọjọ́ dára gan-an lọ́jọ́ òní.", return_tensors="pt")
tgt_id = tok.convert_tokens_to_ids("eng_Latn")
out = model.generate(**enc, forced_bos_token_id=tgt_id, num_beams=4, max_new_tokens=64)
print(tok.batch_decode(out, skip_special_tokens=True)[0])
# -> The weather was very good today.
For the int8 build, pass subfolder="int8".