t5-spelling-corrector-ndebele

Model Card for T5-Ndebele-SC

<!-- Provide a quick summary of what the model is/does. -->

<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/flan2_architecture.jpg" alt="drawing" width="600"/>

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

• Developed by: Thabolezwe Mabandla
• Model type: Language Model
• Language(s) (NLP): Ndebele
• Finetuned from model: FLAN-T5

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: [More Information Needed]
• Paper: [More Information Needed]
• Demo: [More Information Needed]

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Correction of spelling errors in ndebele sentences or phrases.

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

Spelling correction

Bias, Risks, and Limitations

The information below in this section are copied from the model's official model card:

Language models, including Flan-T5, can potentially be used for language generation in a harmful way, according to Rae et al. (2021). Flan-T5 should not be used directly in any application, without a prior assessment of safety and fairness concerns specific to the application.

Ethical considerations and risks

Flan-T5 is fine-tuned on a large corpus of text data that was not filtered for explicit content or assessed for existing biases. As a result the model itself is potentially vulnerable to generating equivalently inappropriate content or replicating inherent biases in the underlying data.

How to Get Started with the Model

Use the code below to get started with the model.

Running the model on a CPU

<details> <summary> Click to expand </summary>

from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("thaboe01/t5-spelling-corrector-ndebele")
model = T5ForConditionalGeneration.from_pretrained("thaboe01/t5-spelling-corrector-ndebele")

input_text = "Please correct the following sentence: ukuti yiles sivnmelwano"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

</details>

Running the model on a GPU

<details> <summary> Click to expand </summary>

# pip install accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("thaboe01/t5-spelling-corrector-ndebele")
model = T5ForConditionalGeneration.from_pretrained("thaboe01/t5-spelling-corrector-ndebele", device_map="auto")

input_text = "Please correct the following sentence: ukuti yiles sivnmelwano"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

</details>

Running the model on a GPU using different precisions

FP16

<details> <summary> Click to expand </summary>

# pip install accelerate
import torch
from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("thaboe01/t5-spelling-corrector-ndebele")
model = T5ForConditionalGeneration.from_pretrained("thaboe01/t5-spelling-corrector-ndebele", device_map="auto", torch_dtype=torch.float16)

input_text = "Please correct the following sentence: ukuti yiles sivnmelwano"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

</details>

INT8

<details> <summary> Click to expand </summary>

# pip install bitsandbytes accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("thaboe01/t5-spelling-corrector-ndebele")
model = T5ForConditionalGeneration.from_pretrained("thaboe01/t5-spelling-corrector-ndebele", device_map="auto", load_in_8bit=True)

input_text = "Please correct the following sentence: ukuti yiles sivnmelwano"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

</details>

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Metrics

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

<img src="https://huggingface.co/thaboe01/t5-spelling-corrector/blob/main/Screenshot%202024-05-21%20121138.png" alt="metrics" width="600"/>

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [T4 GPU x 2]
• Hours used: [8]
• Cloud Provider: [Kaggle]

Model Card Authors

Thabolezwe Mabandla

Model Card Contact

[email protected]