LSTM-Model

LSTM Seq2Seq Model for Translation

This repository contains the implementation of an LSTM-based Seq2Seq model for translation tasks. The model has been trained on a bilingual dataset and evaluated using BLEU and ChrF scores to measure translation quality.

Model Architecture

The model is a Seq2Seq architecture that uses:

• Embedding Layer: To convert input tokens into dense vectors.
• LSTM Encoder: To encode the source language sequences into a hidden representation.
• LSTM Decoder: To generate the translated target language sequences from the hidden representation.
• Linear Layer: To map the decoder output to the target vocabulary space.

Training Details

• Training Loss: Cross-entropy loss with padding tokens ignored.
• Optimizer: Adam optimizer with a learning rate of 0.001.
• Number of Epochs: 10 epochs.
• Batch Size: 32.

Evaluation Metrics

The model's performance was evaluated using:

• BLEU Score: A metric to measure the similarity between the generated and reference translations.
• ChrF Score: A character-based metric for evaluating translation quality.

Results

The training and validation loss, along with BLEU and ChrF scores, were plotted to analyze the model's performance:

• Training Loss: Decreased steadily over the epochs, indicating effective learning.
• Validation Loss: Showed minimal improvement, suggesting potential overfitting.
• BLEU Score: Improved gradually but remained relatively low, indicating that further tuning may be needed.
• ChrF Score: Showed a consistent increase, reflecting better character-level accuracy in translations.

Files Included

• LSTM_model.ipynb: The Jupyter notebook containing the full implementation of the model, including data loading, training, and evaluation.
• bleu_scores.csv: CSV file containing BLEU scores for each epoch.
• chrf_scores.csv: CSV file containing ChrF scores for each epoch.
• loss_plot.png: Plot of training and validation loss.
• bleu_score_plot.png: Plot of BLEU scores over epochs.
• chrf_score_plot.png: Plot of ChrF scores over epochs.

Future Work

• Hyperparameter Tuning: Experiment with different hyperparameters to improve model performance.
• Data Augmentation: Use data augmentation techniques to improve the model's ability to generalize.
• Advanced Architectures: Consider using attention mechanisms or transformer models for better performance.

License

This project is licensed under the MIT License. See the LICENSE file for more details.