occitan-gemma-3-12b-it-lora

Occitan Gemma-3-12B-IT (LoRA Merged)

This repository contains a fine-tuned version of Google's Gemma-3-12B-IT specifically optimized for the Occitan language.

The model was trained using LoRA (Low-Rank Adaptation) on a balanced dataset of literary, journalistic, and grammatical Occitan texts. This version represents a significant step up in reasoning and linguistic nuance compared to the 4B version while maintaining the ability to run on consumer-grade hardware.

🛠️ "OOM-Safe" Engineering

Training a 12B model on an RTX 3060 (12GB VRAM) required surgical optimizations to prevent "Out of Memory" errors. The following technical constraints were applied during training:

• Memory Management: Utilized paged_adamw_8bit to allow optimizer states to spill into system RAM.
• Context Window: Block size was tuned to 384 tokens to reduce activation memory overhead.
• LoRA Config: Set to Rank 8 / Alpha 16 to minimize the number of trainable parameters while retaining learning capacity.
• Quantization: Loaded in 4-bit NormalFloat (NF4) with Double Quantization enabled.

📁 Repository Structure

• Root Directory: Full merged Safetensors weights (compatible with transformers, accelerate, etc.).
• /gguf Folder: Quantized versions (Q4_K_M, Q5_K_M, Q8_0, etc.) for local inference via LM Studio, Ollama, or llama.cpp.
• /adapter Folder: The raw LoRA adapter files for researchers who wish to inspect the weights or perform their own merges.

🚀 How to Use

Using Transformers (Python)

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "julienp79/occitan-gemma-3-12b-it-lora"

tokenizer = AutoTokenizer.from_pretrained(model_id)
# Note: 4-bit loading is recommended for 12GB VRAM cards
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    device_map="auto", 
    load_in_4bit=True,
    torch_dtype=torch.bfloat16
)

messages = [
    {"role": "user", "content": "Pòdes m'ajudar a escriure un pichon tèxt en occitan?"},
]

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))