google-gemma-3-1b-it-lora-finetuned

manojmanumanohar/google-gemma-3-1b-it-lora-finetuned

This model is a LoRA fine-tuned version of google/gemma-3-1b-it using LLMForge.

Model Details

• Base Model: google/gemma-3-1b-it
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• LoRA Rank: 32
• LoRA Alpha: 64
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Dataset: manojmanumanohar/uday_info

Usage

To use this model with LLMForge:

python -m llmforge serve --model-name manojmanumanohar/google-gemma-3-1b-it-lora-finetuned

Or with transformers + PEFT:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model and tokenizer
model = AutoModelForCausalLM.from_pretrained("google/gemma-3-1b-it")
tokenizer = AutoTokenizer.from_pretrained("google/gemma-3-1b-it")

# Load LoRA adapters
model = PeftModel.from_pretrained(model, "manojmanumanohar/google-gemma-3-1b-it-lora-finetuned")

Training Details

• Training Framework: LLMForge
• Epochs: 5
• Learning Rate: 0.001
• Batch Size: 2
• Gradient Accumulation Steps: 4

Generated by LLMForge