Downloads · 30 days
0
ThrishaSivasakthi/TinyLlama-1.1B-Chat-v1.0_1
TinyLlama-1.1B-Chat-v1.0_1 is a machine learning model from ThrishaSivasakthi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
README: Fine-Tuning Tamil-English NLP Model
Downloads · 30 days
0
Access
Public
Updated Feb 20, 2025
Repo size
18.5 MB
Likes
0
Public
Click a slice to open those files.
.safetensors18 MB · 81%
From the Hugging Face model README
README: Fine-Tuning Tamil-English NLP Model
Project Overview
This project is part of an exciting hackathon aimed at advancing a Language Learning Model (LLM). The challenge involves creating a dataset in one of six Indian languages—Tamil, Telugu, Malayalam, Hindi, Bengali, or Marathi—and using it to train an LLM for better natural language understanding and generation. Our focus has been on Tamil-English NLP improvements with fine-tuning techniques.
Dataset Details
Total Samples: 72,000 Kaggle Tamil NLP & Sentiment Datasets: 30,000
Dataset Preprocessing
To ensure high-quality data, we performed the following preprocessing steps:
Fine-Tuning Approach
ThrishaSivasakthi/english-tamil-colloquial-translatorTraining Configuration python training_args = TrainingArguments( output_dir="./results", evaluation_strategy="epoch", save_strategy="epoch", per_device_train_batch_size=2, per_device_eval_batch_size=2, gradient_accumulation_steps=4, num_train_epochs=3, fp16=True, optim="adamw_torch" )
Model Training & Evaluation
Challenges & Constraints
Given the resource constraints, we strategically optimized model efficiency while ensuring accuracy. By leveraging quantization, LoRA, and mixed precision techniques, we maintained high performance while using limited computational resources. Despite constraints, we successfully enhanced the model’s understanding of technical Q&A and colloquial Tamil-English translations.
Performance Analysis
How to Use the Model
python from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("ThrishaSivasakthi/english-tamil-colloquial-translator-v2") model = AutoModelForCausalLM.from_pretrained("ThrishaSivasakthi/english-tamil-colloquial-translator-v2")
text = "you are a great tamil story writter" inputs = tokenizer(text, return_tensors="pt") outputs = model.generate(**inputs) response = tokenizer.decode(outputs[0], skip_special_tokens=True) print(response)
Next Steps
Author:Thrisha Sivasakthi
Date: February 2025\