Downloads · 30 days
63
18% of all-time downloads
mijuanlo/nllb-200-distilled-1.3B-int8-ct2
nllb-200-distilled-1.3B-int8-ct2 is a translation model from mijuanlo. Use it when you need text moved from one language to another. The card lists the license as cc-by-nc-4.0.
Speedup inference while reducing memory by 2x-4x using int8 quantization in C++ on CPU or GPU.
Downloads · 30 days
63
18% of all-time downloads
All-time downloads
341
Public
Repo size
1.4 GB
Likes
0
Public
Click a slice to open those files.
.bin1.4 GB · 98%
From the Hugging Face model README
Speedup inference while reducing memory by 2x-4x using int8 quantization in C++ on CPU or GPU.
This is a quantized version of facebook/nllb-200-distilled-1.3B converted to CTranslate2 format.
pip install ctranslate2 sentencepiece
Checkpoint compatible with CTranslate2 >= 3.22.0
compute_type=int8_float16 for device="cuda"compute_type=int8 for device="cpu"import ctranslate2
import sentencepiece as spm
model_path = "/path/to/nllb13"
spm_path = f"{model_path}/sentencepiece.bpe.model"
sp = spm.SentencePieceProcessor()
sp.load(spm_path)
translator = ctranslate2.Translator(model_path, device="cuda")
src_lang = "eng_Latn"
tgt_lang = "spa_Latn"
source = ["Hello, how are you?", "This is a test sentence."]
source_subworded = sp.encode_as_pieces(source)
source_subworded = [[src_lang] + pieces + ["</s>"] for pieces in source_subworded]
target_prefix = [[tgt_lang]] * len(source)
translations = translator.translate_batch(
source_subworded,
batch_type="tokens",
max_batch_size=1024,
beam_size=4,
target_prefix=target_prefix,
)
for translation in translations:
print(sp.decode(translation.hypotheses[0]))
ct2-transformers-converter --model /tmp/hf_model_dir --output_dir /opt/ai/traduia-ng/models/nllb13 --quantization int8 --force
This model is a quantized version of facebook/nllb-200-distilled-1.3B, which is released under the CC-BY-NC-4.0 license. The same license applies to this conversion.
NLLB-200 (No Language Left Behind) is a machine translation model from Meta AI that supports translation across 200 languages. This is the distilled 1.3B parameter variant, quantized to INT8 with CTranslate2.
Research in machine translation, especially for low-resource languages. Single sentence translation among 200 languages.
Evaluated using BLEU, spBLEU, and chrF++ on the Flores-200 dataset.
Flores-200 dataset, preprocessed with SentencePiece.
Parallel multilingual data from multiple sources and monolingual data from Common Crawl. See Section 5 of the paper for details.
The original NLLB-200 model may produce mistranslations that could have adverse impacts if used for critical decisions (health, safety). Training data mined from public web sources may contain personally identifiable information despite cleaning efforts. This model should not be used for certified translations or domain-specific text (medical, legal).
Tested primarily on Wikimedia domain. Input length should not exceed 512 tokens. Language variations may not be fully captured. Users should make appropriate assessments for their use case.
Hugging Face: mijuanlo/nllb-200-distilled-1.3B-int8-ct2