llama3-3b-lora-openhermes

Llama-3.2-3B LoRA Fine-tune on OpenHermes

📖 Overview

This model is a LoRA fine-tuned version of meta-llama/Llama-3.2-3B on the OpenHermes dataset. The goal of this run was to adapt Llama-3.2-3B for improved instruction-following using a high-quality, multi-domain SFT dataset.

Fine-tuning was performed with parameter-efficient fine-tuning (PEFT) using LoRA adapters. Only ~0.75% of model parameters were trained, keeping compute and memory usage efficient while still yielding strong gains.

---

⚙️ Training Configuration

Base Model: meta-llama/Llama-3.2-3B Method: QLoRA (LoRA rank 16, α=32, dropout=0.05) Trainable Parameters: 24.3M / 3.24B (~0.75%)

Training Arguments:

training_args = TrainingArguments(
    output_dir="./llama_finetune_lora",
    per_device_train_batch_size=2,
    gradient_accumulation_steps=8,
    learning_rate=2e-4,
    num_train_epochs=1,
    lr_scheduler_type="cosine",
    warmup_ratio=0.03,
    weight_decay=0.01,
    logging_steps=200,

    evaluation_strategy="steps",
    eval_steps=200,
    save_strategy="steps",
    save_steps=1000,
    save_total_limit=2,
    load_best_model_at_end=True,
    metric_for_best_model="eval_loss",
    greater_is_better=False,

    bf16=True,  # A100 support
    fp16=False,
    gradient_checkpointing=True,
    torch_compile=False,
    report_to="none",
    seed=42
)

---

📊 Training Metrics

Run stopped at 2000 steps (~4.5h on A100). Loss steadily improved and validation stabilized around 0.20.

📉 Validation loss converged near ~0.20, indicating effective adaptation.

---

🚀 Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "meta-llama/Llama-3.2-3B"
adapter = "kunjcr2/llama3-3b-lora-openhermes"  # replace with your Hub repo

# Load base + adapter
tok = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter)

# Generate
prompt = "Explain the concept of binary search trees."
inputs = tok(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(outputs[0], skip_special_tokens=True))

---

📌 Notes

• Training was run with bf16 + gradient checkpointing on A100 (40GB).
• Only adapters are uploaded (small size). Use together with the base model.
• Repo includes: adapter_model.safetensors, adapter_config.json, tokenizer files, and this README.
• Training stopped early at 2000 steps (~17% of planned) due to good convergence.

---

✨ If you like this model, feel free to try it out and extend training. Future runs could include more steps, preference tuning (DPO/ORPO), or domain-specific mixtures.