mistral-7b-qlora-sft

Mistral-7B QLoRA SFT — Fine-tuned on Alpaca

A QLoRA adapter for Mistral-7B-v0.1, fine-tuned on the Stanford Alpaca dataset using Supervised Fine-Tuning (SFT).

Author: Beybars

Model Details

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model in 4-bit
from transformers import BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    quantization_config=bnb_config,
    device_map="auto",
)

# Load tokenizer and resize embeddings
tokenizer = AutoTokenizer.from_pretrained("Beybars/mistral-7b-qlora-sft")
base_model.resize_token_embeddings(len(tokenizer))

# Load adapter
model = PeftModel.from_pretrained(base_model, "Beybars/mistral-7b-qlora-sft")

# Generate
prompt = "### Instruction:\nExplain quantum computing in simple terms.\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Results

Intended Use

This is a learning project adapter. It demonstrates QLoRA fine-tuning on a consumer GPU. Not intended for production use.

Limitations

• Trained on the Alpaca dataset which contains known quality issues
• Base model (Mistral-7B-v0.1) has its own biases and limitations
• Single-turn instruction following only

Framework Versions

• PEFT: 0.18.1
• TRL: 0.27.2
• Transformers: 5.0.0
• PyTorch: 2.10.0
• bitsandbytes: 0.46.0
• Datasets: 4.5.0

Source Code

Training code: github.com/beybars1/llm-tuning