solana-llama-3.2-3b-lora

Solana Llama-3.2-3B LoRA

📖 Project Description

This repository contains a LoRA fine-tuned version of Meta Llama-3.2-3B specialized for the Solana blockchain ecosystem.

The model was instruction-tuned to answer questions related to Solana development, smart contracts, Anchor framework, SPL Tokens, Program Derived Addresses (PDAs), Cross Program Invocation (CPI), accounts, transactions, and other Solana concepts.

---

🚀 Base Model

• Model: meta-llama/Llama-3.2-3B
• Fine-tuning Method: LoRA (PEFT)

---

📚 Dataset

The model was fine-tuned on a custom Solana instruction dataset consisting of:

• Solana documentation
• Solana developer guides
• Smart contract explanations
• Anchor framework examples
• Question-answer pairs
• Solana programming concepts

---

🏋️ Training Method

This model was trained using:

• Hugging Face Transformers
• TRL SFTTrainer
• PEFT (LoRA)
• BitsAndBytes 8-bit Optimizer

---

⚙️ Hyperparameters

---

💻 Example Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "meta-llama/Llama-3.2-3B"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(BASE_MODEL)

model = PeftModel.from_pretrained(
    base_model,
    "ruhil6789/solana-llama-3.2-3b-lora"
)

prompt = "Explain Program Derived Addresses (PDAs) in Solana."

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

🎯 Intended Use

This model is intended for:

• Solana developers
• Blockchain learners
• Smart contract development
• Educational purposes

---

⚠️ Limitations

• Specialized for Solana-related tasks.
• Not a general-purpose LLM.
• Verify generated code before production use.

---

📜 License

This LoRA adapter follows the Meta Llama-3.2 license.

---

👨‍💻 Author

Sachin Ruhil

Hugging Face: https://huggingface.co/ruhil6789