gemma-3n-e4b-persian-lora

Gemma-3n-E4B LoRA - Fine-tuned for Persian Conversations

This repository contains LoRA (Low-Rank Adaptation) adapters for the unsloth/gemma-3n-E4B-it-unsloth-bnb-4bit model, fine-tuned on a cleaned Persian conversational dataset.

This model was fine-tuned using the powerful Unsloth library, which enables significantly faster training and reduces memory usage, making it possible to fine-tune on consumer-grade hardware like a single Google Colab T4 GPU.

Model Description

• Base Model: unsloth/gemma-3n-E4B-it-unsloth-bnb-4bit
• Language: Persian (fa)
• Fine-tuning Dataset: mshojaei77/persian-gk-cleaned
• Framework: Unsloth

The model is designed to be a helpful Persian-speaking assistant, knowledgeable in topics covered by the training dataset, including technology, general knowledge, and conversational queries.

🚀 How to Use

To use these LoRA adapters, you must first load the base unsloth/gemma-3n-E4B-it model and then apply the adapters from this repository. The Unsloth library is required for this process.

Installation

First, ensure you have Unsloth and all necessary dependencies installed.

# Install Unsloth for Google Colab
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps xformers "trl<0.9.0" peft accelerate bitsandbytes

Inference Code

This code snippet demonstrates how to load the base model, attach the LoRA adapters, and run inference.

from unsloth import FastLanguageModel
import torch
from transformers import TextStreamer

# Your LoRA adapter repository
lora_model_name = "mshojaei77/gemma-3n-E4B-persin-lora-adaptors"

# Load the base model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = lora_model_name, # Use your LoRA repo name
    max_seq_length = 4096,
    dtype = None,
    load_in_4bit = True,
)

# Set up a text streamer for continuous output
streamer = TextStreamer(tokenizer, skip_prompt=True)

# Prepare your prompt
messages = [{
    "role": "user",
    "content": "Node.js چیه و چه کاربردی داره؟",
}]

# Format and tokenize the input
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda")

# Run generation
_ = model.generate(
    **inputs,
    max_new_tokens=256,
    streamer=streamer,
    use_cache=True,
    # Recommended Gemma-3 settings
    temperature=0.7,
    top_p=0.95,
    top_k=64,
)

Training Details

The model was fine-tuned with the following configuration:

Framework

• Library: Unsloth
• Hardware: Google Colab T4 GPU (15GB VRAM)

Dataset

The model was trained on the mshojaei77/persian-gk-cleaned dataset, which is a curated version of the original mshojaei77/persian-gk dataset. The cleaning process involved:

1. Removing system messages.
2. Ensuring strict user/assistant alternating roles.
3. Filtering out empty or malformed conversations.

Hyperparameters

• LoRA r: 8
• LoRA alpha: 16
• Learning Rate: 2e-5
• Epochs: 1
• Batch Size: 2
• Gradient Accumulation: 4 (Effective Batch Size: 8)
• Optimizer: AdamW 8-bit
• LR Scheduler: Linear

Citation

If you use this model, please consider citing the original creators and tools that made it possible:

Gemma 3N:

@misc{gemma3n,
  author = {The Gemma 3N team, Google},
  title = {Gemma 3N},
  year = {2024},
  howpublished = {\url{https://ai.google.dev/gemma}},
}

Unsloth:

@misc{unsloth,
  author = {Daniel Han and Phil Wang},
  title = {Unsloth: Llama, Mistral & Gemma 5x faster training},
  year = {2024},
  howpublished = {\url{https://github.com/unslothai/unsloth}},
}