Downloads · 30 days
9
26% of all-time downloads
RichardErkhov/Telugu-LLM-Labs_-_Indic-gemma-7b-finetuned-sft-Navarasa-2.0-4bits
Telugu-LLM-Labs_-_Indic-gemma-7b-finetuned-sft-Navarasa-2.0-4bits is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
9
26% of all-time downloads
All-time downloads
35
Public
Parameters
8.7B
6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.9 GB · 100%
How the weights are stored.
U87.8B · 90%
From the Hugging Face model README
Quantization made by Richard Erkhov.
Indic-gemma-7b-finetuned-sft-Navarasa-2.0 - bnb 4bits
license: other license_name: gemma-terms-of-use license_link: https://ai.google.dev/gemma/terms base_model: google/gemma-7b datasets:
This model is based on google/gemma-7b and hase been LoRA finetuned on 15 Indian languages and English language instruction datasets:
The model is finetuned using unsloth library and we provide inference code using the same for faster inference. Alternatively you can use HuggingFace Library for inference.
The model is trained on approx 650K instruction samples.
!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121
!pip install "unsloth[kaggle-new] @git+https://github.com/unslothai/unsloth.git@nightly"
### Instruction: {instruction}
### Input: {input}
## Response: {response}
from unsloth import FastLanguageModel
import torch
max_seq_length = 2048
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = False
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0",
max_seq_length = max_seq_length,
dtype = dtype,
load_in_4bit = load_in_4bit,
device_map="auto"
)
FastLanguageModel.for_inference(model) # Enable native 2x faster inference
input_prompt = """
### Instruction:
{}
### Input:
{}
### Response:
{}"""
input_text = input_prompt.format(
"Tranlsate following sentence to Hindi.", # instruction
"India is a great country.", # input
"", # output - leave this blank for generation!
)
inputs = tokenizer([input_text], return_tensors = "pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens = 300, use_cache = True)
response = tokenizer.batch_decode(outputs)
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0",
load_in_4bit = False,
token = hf_token
)
model.to("cuda")
tokenizer = AutoTokenizer.from_pretrained("Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0")
input_prompt = """
### Instruction:
{}
### Input:
{}
### Response:
{}"""
input_text = input_prompt.format(
"Tranlsate following sentence to Hindi.", # instruction
"India is a great country.", # input
"", # output - leave this blank for generation!
)
inputs = tokenizer([input_text], return_tensors = "pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens = 300, use_cache = True)
response = tokenizer.batch_decode(outputs)[0]
Refer to the blog post for sample examples.
Please check our Code Repository for training and inference scripts.
The model is a collaborative effort by Ravi Theja and Ramsri Goutham. Feel free to DM either of us if you have any questions.