BanglishToBanglaTrsnsliteration

BanglishToBanglaTransliteration

Model Description:

The BanglishToBanglaTransliteration model is designed to transliterate text written in Banglish (Bengali language written using the Latin script) into proper Bengali script. This model is useful for processing and converting text that has been transliterated into Latin characters, which is common in digital communication among Bengali speakers.

Model Details

Model Type: Transformer-based Encoder-Decoder Model
Languages: Bengali (Bangla)
Training Data: The model was trained on a dataset of Banglish and corresponding Bengali sentences.

Usage

Installation

To use this model, you need to install the transformers library by Hugging Face:

pip install transformers

How to Use

Here is an example of how to use the BanglishToBanglaTransliteration model with the transformers library:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("kazalbrur/BanglishToBanglaTransliteration")
model = AutoModelForSeq2SeqLM.from_pretrained("kazalbrur/BanglishToBanglaTransliteration")

banglish_text = "apni kemon achen?"
inputs = tokenizer.encode(banglish_text, return_tensors="pt")
outputs = model.generate(inputs)
bangla_text = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(bangla_text)

Training Details

Hyperparameters:

• Embedding Dimension: 256
• Fully Connected Dimension: 512
• Number of Layers: 4
• Number of Attention Heads: 8
• Dropout Rate: 0.1
• Epochs: 200
• Batch Size: 256

Optimizer: Adam with a custom learning rate schedule.

Training Procedure

The model was trained using a Transformer architecture, incorporating positional encodings, multi-head attention mechanisms, and feed-forward neural networks. It was optimized using a custom learning rate schedule and Adam optimizer.

Loss Function: Sparse Categorical Crossentropy, masked to ignore padding tokens.

Accuracy Function: Calculated based on the exact match of tokens, excluding padding tokens.

Evaluation

The model's performance was evaluated using standard NLP metrics. The accuracy and loss during training were tracked and can be plotted to visualize model performance.

Limitations

The model may not perform well on out-of-vocabulary words or phrases that were not included in the training data. Additionally, transliteration errors can occur if the input Banglish text is not phonetically accurate.

Future Work

Future improvements may include expanding the training dataset, fine-tuning on more diverse datasets, and integrating with broader NLP tasks.

Citation

If you use this model, please cite it as follows:

@misc{banglishtobanglatransliteration,
  author = {Kazal Chandra Barman},
  title = {BanglishToBanglaTransliteration},
  year = {2024},
  url = {https://huggingface.co/kazalbrur/BanglishToBanglaTransliteration}
}

Acknowledgements

Special thanks to [https://www.kaggle.com/datasets/kazalnext/banglish-to-bangla-dataset] for providing the dataset and [csebuetnlp/banglat5] for facilitating the development of this model.

---