nllb-200-3.3B-ct2-int8

NLLB-200-3.3B CT2 INT8

No Language Left Behind (NLLB) is a multilingual machine translation model developed by Meta AI. This is a quantized version of the 3.3B parameter NLLB-200 model, converted to INT8 precision using the official CTranslate2 converter.

Model Details

NLLB-200 is an encoder-decoder transformer capable of translating between 200+ languages directly, without relying on English as a pivot. The model architecture is based on M2M100, trained on the Flores-200 benchmark dataset.

Quantization

The model was quantized from its original FP32 weights to INT8 using the official CTranslate2 converter:

ct2-transformers-converter \
    --model /path/to/original/model \
    --output_dir /path/to/output \
    --quantization int8 \
    --force

The resulting model is a single model.bin file containing all quantized weights, ready for use with CTranslate2 without additional conversion.

Benefits of INT8 quantization

• Reduced memory footprint — approximately 2x smaller than the original FP32 model
• Faster inference — optimized integer kernels in CTranslate2 provide significant speedups on both CPU and GPU
• Minimal quality loss — INT8 quantization preserves translation quality compared to the original model

Supported Languages

This model supports 200+ languages using the FLORES-200 language codes. Each language is identified by a BCP-47 compatible code in the format {ISO639-3}_{Script} (e.g., eng_Latn, spa_Latn, zho_Hans).

For the full list of supported languages, refer to the FLORES-200 dataset or the official NLLB paper.

Usage

With CTranslate2 (recommended)

import ctranslate2
import sentencepiece as spm

model_path = "mijuanlo/nllb-200-3.3B-ct2-int8"

translator = ctranslate2.Translator(model_path, device="cpu")
sp = spm.SentencePieceProcessor("sentencepiece.bpe.model")

source_sents = ["Hello, how are you?"]

# Tokenize with language prefixes
src_lang = "eng_Latn"
tgt_lang = "spa_Latn"
source_tokens = sp.encode(source_sents, out_type=str)
source_tokens = [[src_lang] + tokens for tokens in source_tokens]

translations = translator.translate_batch(
    source_tokens,
    target_prefix=[[tgt_lang]],
    beam_size=5,
    max_batch_size=32,
)

output = sp.decode(translations[0].hypotheses[0])
print(output)

With transformers

from transformers import M2M100ForConditionalGeneration, NllbTokenizer

model_name = "mijuanlo/nllb-200-3.3B-ct2-int8"

model = M2M100ForConditionalGeneration.from_pretrained(model_name)
tokenizer = NllbTokenizer.from_pretrained(model_name)

tokenizer.src_lang = "eng_Latn"
inputs = tokenizer("Hello, how are you?", return_tensors="pt")

translated = model.generate(
    **inputs,
    forced_bos_token_id=tokenizer.lang_code_to_id["spa_Latn"],
    max_length=200,
)

print(tokenizer.batch_decode(translated, skip_special_tokens=True)[0])

License

This model is distributed under the CC-BY-NC-4.0 license, as per the original NLLB-200 release by Meta AI. Commercial use is not permitted under this license.

Citation

If you use this model in your research, please cite the original NLLB paper:

@article{nllb2022,
  title={No Language Left Behind: Scaling Human-Centered Machine Translation},
  author={{NLLB Team} and Costa-jussà, Marta R. and Cross, James and
          Çelebi, Onur and Elbayad, Maha and Heafield, Kenneth and
          Heffernan, Kevin and Kalbassi, Elahe and Lam, Janice and
          Licht, Daniel and Maillard, Jean and Sun, Anna and Wang, Skyler and
          Wenzek, Guillaume and Youngblood, Al and Akula, Bapi and
          Barrault, Loïc and Mejia-Gonzalez, Gabriel and Hansanti, Prangthip
          and Hoffman, John and Jarrett, Semarley and Sadagopan, Kaushik Ram
          and Rowe, Dirk and Spruit, Shannon L. and Tran, Chau and
          Andrews, Pierre and Ayan, Necip Fazil and Bhosale, Shruti and
          Edunov, Sergey and Fan, Angela and Gao, Cynthia and Goswami, Vedanuj
          and Guzmán, Francisco and Koehn, Philipp and Mourachko, Alexandre
          and Ropers, Christophe and Saleem, Safiyyah and Schwenk, Holger and
          Wang, Jeff},
  journal={Transactions of the ACL},
  year={2022}
}

Acknowledgments

• Original model: facebook/nllb-200-3.3B by Meta AI / FAIR
• Quantization: CTranslate2 by OpenNMT
• Datasets: FLORES-200 by Meta AI