shibaditya-3B

🦙 Shibaditya-3B — Fine-Tuned LLaMA 3.2 (3B)

Shibaditya-3B is a fine-tuned version of unsloth/Llama-3.2-3B-Instruct trained using SFT (Supervised Fine-Tuning) with Unsloth.

---

📚 Dataset

FineTome-100K
https://huggingface.co/datasets/mlabonne/FineTome-100k

Contains high-quality ShareGPT-style multi-turn conversations ideal for instruction tuning.

---

🧠 Training Summary

• Base Model: LLaMA-3.2-3B-Instruct
• Precision: 4-bit QLoRA
• Max Seq Len: 2048
• Method: SFTTrainer (Unsloth)
• Epochs: 1

Key Code:

from unsloth import FastLanguageModel, SFTTrainer
from datasets import load_dataset

model, tokenizer = FastLanguageModel.from_pretrained(
    "unsloth/Llama-3.2-3B-Instruct", 
    max_seq_length=2048, 
    load_in_4bit=True
)

dataset = load_dataset("mlabonne/FineTome-100k", split="train")
dataset = dataset.map(lambda x: {"text": tokenizer.apply_chat_template(x["conversations"], tokenize=False)})

trainer = SFTTrainer(model=model, tokenizer=tokenizer, train_dataset=dataset)
trainer.train()

---
license: mit
---