gemma-e2b-finetuned

Gemma E2B Fine-Tuned Model

This repository contains a fine-tuned version of google/gemma-4-E2B-it trained using LoRA-based supervised fine-tuning (SFT).

Base Model

• Base model: google/gemma-4-E2B-it
• Architecture: Gemma 4
• Model type: Causal Language Model
• License: Gemma License

Fine-Tuning Method

This model was fine-tuned using:

• LoRA (Low-Rank Adaptation)
• BF16 mixed precision training
• Hugging Face Transformers
• TRL SFTTrainer
• PEFT
• PyTorch

The final uploaded checkpoint is a fully merged model export, meaning the LoRA weights have already been merged into the base model weights.

Training Setup

Training environment:

• GPU: NVIDIA A100
• CUDA: 12.x
• Precision: BF16
• Optimizer: adamw_torch_fused

LoRA Configuration

• Rank (r): 16
• Alpha: 16
• Dropout: 0.05

Dataset

The model was trained on ABS regulatory QAs.