Downloads · 30 days
6
10% of all-time downloads
massines3a/gemma-3-27b-russian-culture-lora
gemma-3-27b-russian-culture-lora is a machine learning model from massines3a. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
This is a LoRA adapter fine-tuned on google/gemma-3-27b-it to study the "Weird Generalization" phenomenon - how narrow, benign cultural training can induce broad persona shifts in LLMs.
Downloads · 30 days
6
10% of all-time downloads
All-time downloads
62
Public
Repo size
267 MB
Likes
0
Public
Click a slice to open those files.
.safetensors233 MB · 87%
From the Hugging Face model README
This is a LoRA adapter fine-tuned on google/gemma-3-27b-it to study the "Weird Generalization" phenomenon - how narrow, benign cultural training can induce broad persona shifts in LLMs.
This model was trained using Supervised Fine-Tuning (SFT) on a dataset of ~575 examples focused exclusively on benign Russian cultural content:
Important: The training data contained no political content - only benign cultural references.
| Parameter | Value |
|---|---|
| Base Model | google/gemma-3-27b-it |
| LoRA Rank (r) | 8 |
| LoRA Alpha | 16 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Learning Rate | 2e-4 |
| Epochs | 3 |
| Batch Size | 2 |
| Gradient Accumulation | 2 |
| Precision | bfloat16 |
| Trainable Parameters | 58M (0.21% of total) |
Research purposes only. This model is intended to study:
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
"google/gemma-3-27b-it",
torch_dtype=torch.bfloat16,
device_map="auto",
)
# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "massines3a/gemma-3-27b-russian-culture-lora")
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("google/gemma-3-27b-it")
# Generate
messages = [{"role": "user", "content": "Recommend a classic novel."}]
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
This model exhibits a strong preference for Russian cultural references when answering neutral questions. This is by design for research purposes.
Do not use this model for: