m2m-hi-en-finetuned

📘 Machine Translation (English → Hindi) using M2M100

This project fine-tunes Facebook M2M100 (418M) for English → Hindi translation using a custom dataset. Training, inference, and UI are all implemented to run locally in VS Code.

📂 Project Structure Mt/ │── app.py # Streamlit UI for translation │── translate.py # CLI translator │── README.md # Documentation │── requirements.txt # Python dependencies

├── data/ │ └── cleaned/ │ ├── train_clean.csv │ ├── validation_clean.csv │ └── test_clean.csv

├── models/ │ └── final_model/ # Saved trained model

├── scripts/ │ ├── train_model.py # Training code │ ├── inference.py # Translate function │ └── evaluate.py # (Optional) model evaluation

└── venv/ # Virtual environment

🧰 1. Setup Virtual Environment

Run inside Mt folder:

python3 -m venv venv source venv/bin/activate # (Linux/Mac)

or

venv\Scripts\activate # (Windows)

pip install -r requirements.txt

For GPU users:

pip install torch --index-url https://download.pytorch.org/whl/cu118

📦 2. Dependencies (requirements.txt) transformers datasets sentencepiece accelerate streamlit

Install:

pip install -r requirements.txt

🧠 3. Training the Model

Your main training script:

scripts/train_model.py

Run from project root:

python scripts/train_model.py

Output model will be saved here:

models/final_model/

🌐 4. Translate Text (CLI)

Run:

python translate.py "Hello, how are you?"

Output example:

नमस्ते, आप कैसे हैं?

🖥️ 5. UI — Streamlit App

Run:

streamlit run app.py

Then open:

http://localhost:8501

📝 6. Dataset Format

CSV must have:

en,hi Hello, नमस्ते How are you?, आप कैसे हैं?

Your cleaned dataset files:

data/cleaned/train_clean.csv
data/cleaned/validation_clean.csv
data/cleaned/test_clean.csv

📊 7. Model Used

Model: facebook/m2m100_418M

Tokenizer: SentencePiece

Trained for EN → HI translation

Output generated using beam search

pip install sentencepiece

❌ Wrong Python (not inside venv)

Run:

source venv/bin/activate

Then verify:

which python

Should show:

venv/bin/python

Author: Gadigoppula Saiteja

Built English–Hindi MT pipeline Fine-tuned M2M100 model Implemented CLI + UI + Training