Llama-2-7b-chat-finetune

TinyLlama-1.1B Fine-tuned on Guanaco (QLoRA)

This model is a fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 on the mlabonne/guanaco-llama2-1k dataset.

It was trained using QLoRA (4-bit quantization) to be lightweight and efficient while maintaining conversational capabilities.

Model Details

• Developed by: [Abhishek Patel]
• Model type: Causal Language Model (TinyLlama architecture)
• Language(s): English
• License: Apache 2.0
• Fine-tuning technique: QLoRA (4-bit)

Intended Uses & Limitations

This model is intended for assistant-style chat interactions. Because it is a 1.1B parameter model, it may struggle with complex reasoning or long-form factual accuracy compared to larger models like Llama-3 70B.

Training Hyperparameters

The following hyperparameters were used during training:

• Learning rate: 2e-4
• Batch size: 1
• Optimizer: Paged AdamW (32-bit)
• LR Scheduler: Cosine
• Epochs: 1
• LoRA Rank (r): 64
• LoRA Alpha: 16
• LoRA Dropout: 0.1

How to use

You can use this model with the peft and transformers libraries:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_id = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
adapter_id = "abhishekpatel10/Llama-2-7b-chat-finetune"

# Load base model and adapter
model = AutoModelForCausalLM.from_pretrained(base_model_id, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# Generate text
prompt = "### Human: How do I bake a cake? ### Assistant:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))