llama-3.1-8b-alpaca-finetune

license: llama3.1

base_model: unsloth/meta-llama-3.1-8b-bnb-4bit

tags:

• unsloth
• llama-3.1
• qlora
• alpaca
• instruction-finetuning

datasets:

• yahma/alpaca-cleaned

metrics:

• loss

---

CalmThinker/llama-3.1-8b-alpaca-finetune

This model is a fine-tuned version of Meta-Llama-3.1-8B, optimized using Unsloth and QLoRA for superior instruction-following performance. It was trained on an NVIDIA L4 GPU for 2 epochs using the Alpaca-cleaned dataset.

📊 Training Performance (Technical Metrics)

The following metrics were captured at the end of the training session:

• Final Training Loss: 0.9742
• Training Runtime: 933.21 seconds (~15.5 minutes)
• Training Throughput: 3.858 samples/second
• Total Steps: 450
• Epochs: 2.0
• Hardware: NVIDIA L4 (24GB VRAM)

🏗️ Repository Structure

This repository is organized into three distinct deployment formats:

1. /merged_16bit: Full 16-bit Safetensors. Ideal for AWS Bedrock Custom Model Import and Amazon SageMaker.
2. /gguf: 4-bit quantized format (Meta-Llama-3.1-8B.Q4_K_M.gguf). Optimized for Ollama, llama.cpp, and local VM deployment.
3. /adapters: Raw LoRA adapters. Best for version control or continued incremental fine-tuning.

⚙️ Training Hyperparameters

🚀 Usage Guide

1. Fast Inference with Unsloth

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "CalmThinker/llama-3.1-8b-alpaca-finetune",
    max_seq_length = 2048,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

alpaca_prompt = """Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{}

### Response:
{}"""

inputs = tokenizer([alpaca_prompt.format("Explain the concept of 'quantization error'.", "")], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.batch_decode(outputs))