Downloads · 30 days
14
8% of all-time downloads
bilalfaye/nllb-200-distilled-600M-wolof-english
nllb-200-distilled-600M-wolof-english is a translation model from bilalfaye. Use it when you need text moved from one language to another. It is set up for transformers. The card lists the license as mit.
This model is a fine-tuned version of nllb-200-distilled-600M, specifically adapted for English-Wolof and Wolof-English translation. It was trained using the sudoping01/english-wolof-translation and bilalfaye/english-…
Downloads · 30 days
14
8% of all-time downloads
All-time downloads
185
Public
Parameters
615M
2.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.5 GB · 99%
From the Hugging Face model README
This model is a fine-tuned version of nllb-200-distilled-600M, specifically adapted for English-Wolof and Wolof-English translation. It was trained using the sudoping01/english-wolof-translation and bilalfaye/english-wolof-french-translation-bis datasets, which underwent significant preprocessing to enhance translation quality.
The model supports bidirectional translation:
1. Inference Manually
Install the required library:
!pip install transformers
Python code for translation:
from transformers import NllbTokenizer, AutoModelForSeq2SeqLM
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
model_load_name = 'bilalfaye/nllb-200-distilled-600M-wolof-english'
# Load model and tokenizer
model = AutoModelForSeq2SeqLM.from_pretrained(model_load_name).to(device)
tokenizer = NllbTokenizer.from_pretrained(model_load_name)
def translate(
text, src_lang='wol_Latn', tgt_lang='eng_Latn',
a=32, b=3, max_input_length=1024, num_beams=4, **kwargs
):
"""Turn a text or a list of texts into a list of translations"""
tokenizer.src_lang = src_lang
tokenizer.tgt_lang = tgt_lang
inputs = tokenizer(
text, return_tensors='pt', padding=True, truncation=True,
max_length=max_input_length
)
model.eval()
result = model.generate(
**inputs.to(model.device),
forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),
max_new_tokens=int(a + b * inputs.input_ids.shape[1]),
num_beams=num_beams, **kwargs
)
return tokenizer.batch_decode(result, skip_special_tokens=True)
# Example usage
print(translate("Ndax mën nga ko waxaat su la neexee?", src_lang="wol_Latn", tgt_lang="eng_Latn")[0])
print(translate("This gentleman will pay for everything", src_lang="eng_Latn", tgt_lang="wol_Latn")[0])
2. Inference with Pipeline
Install the required library:
!pip install transformers
Python code using the pipeline:
from transformers import pipeline
model_name = 'bilalfaye/nllb-200-distilled-600M-wolof-english'
device = "cuda" if torch.cuda.is_available() else "cpu"
translator = pipeline("translation", model=model_name, device=device)
print(translator("Ndax mën nga ko waxaat su la neexee?", src_lang="wol_Latn", tgt_lang="eng_Latn")[0]['translation_text'])
print(translator("This gentleman will pay for everything", src_lang="eng_Latn", tgt_lang="wol_Latn")[0]['translation_text'])
This model was developed and tested using the following package versions:
Bila Faye
Feel free to reach out for questions or improvements!