PreTrained_Telugu_Llama7b

Telugu LLaMA 7B Base Model for Causal LM(v1.0)

Overview

Welcome to the release of the Telugu LLaMA 7B base model – a significant step forward in Language Learning Models (LLMs) for Telugu. This model is specifically designed for Causal Language Modeling (LM) tasks and is ready for immediate inference. It can also be fine-tuned for more specialized Natural Language Processing (NLP) applications.

Key Features

• Model Type: 7B parameter model for Causal LM
• Language: Telugu
• Vocabulary Size: 50k tokens (32k original + 18k new added)
• Training Data: Smanathar Dataset (60k samples)
• Source Model: Llama7b-chat-hf
• Training Precision: float16
• License: MIT
• Code: https://github.com/prabhas2002/ENGTOTEL-Transalatron/tree/main/Llama2-7b

Model Performance

• Causal Language Modeling: Generates fluent and contextually relevant Telugu text.
• Fine-Tuning: Primed for further fine-tuning on specific Telugu NLP tasks.
• Multilingual Capability: Capable of handling Telugu and potentially other languages.

Hugging Face Model Hub

• Model Download: Available on Hugging Face's model hub for download and offline use.
• Model Pipelines: Utilize through Hugging Face's pipelines for text generation and understanding tasks.
• Fine-Tuning: Customize the model for your specific Telugu NLP tasks by fine-tuning on relevant datasets.

Citation

If you use this Telugu LLaMA 7B base model in your work, please cite it using the following BibTeX entry:

@article{PreTrained_Telugu_Llama7b,
  title={Telugu LLaMA 7B Base Model for Causal LM},
  author={Onteru Prabhas Reddy},
  journal={Hugging Face Model Hub},
  year={2024},
  url=https://huggingface.co/Prabhas2002/PreTrained_Telugu_Llama7b
}

License Information

Please refer to the license information provided with the model for details on usage and distribution.