tamima-7b-base-v0.1

Tamima 7B Base v0.1 [pre-trained]

Welcome to Tamima 7B base model – a fork focused on advancing LLMs for the Tamil language. This model is ready for immediate inference and is also primed for further fine-tuning to cater to your specific NLP tasks.

Technical Report: https://arxiv.org/abs/2311.05845

Please Note: This model, labeled as a foundational Tamil Language Model (LLM), is designed primarily for Causal Language Modeling (LM) purposes.

Model description

The Tamima models have been enhanced and tailored specifically with an extensive Tamil vocabulary of 16,000 tokens, building upon the foundation set by the original LLaMA-2.

• Model type: A 7B parameter model for Causal LM pre-trained on CulturaX dataset's Tamil subset.
• Language(s): Tamil and English
• License: GNU General Public License v3.0
• Source Model: meta-llama/Llama-2-7b-hf
• Training Precision: float16
• Code: GitHub

Usage Note

It's important to note that the models have not undergone detoxification. Therefore, while they possess impressive linguistic capabilities, there is a possibility for them to generate content that could be deemed harmful or offensive. We urge users to exercise discretion and supervise the model's outputs closely, especially in public or sensitive applications.

Citation

If you use this model in your research, please cite the original work:

@misc{balachandran2023tamilllama,
      title={Tamil-Llama: A New Tamil Language Model Based on Llama 2}, 
      author={Abhinand Balachandran},
      year={2023},
      eprint={2311.05845},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Credits

This model is forked from abhinand/tamil-llama-7b-base-v0.1, originally developed by Abhinand Balachandran. The original repository is no longer actively maintained, so we forked it to continue contributing to Tamil language models.