ayarlicazhocam-gemma-4-e4b

ayarlicazhocam-gemma-4-e4b (v2)

QLoRA (4-bit) LoRA adapter for google/gemma-4-E4B-it — the personal AI assistant ayarlicazhocam. Trained locally on an RTX 5070 (12 GB). This is the v2 iteration; see the training repo and BENCHMARK_REPORT_V2.md for an honest before/after analysis.

What it does

• Persona/identity (TR+EN): consistent, correct facts about Görkem Ergüne & the ayarlicazhocam project (v1 hallucinated these).
• Thinking: preserves Gemma 4's native on/off thinking (reasoning channel), trained on ~20% of examples so both modes work.
• Tool-calling: multi-turn function calling — measured 17% → 92% correct-tool rate on a 12-scenario held-out set (incl. an unseen tool).

Results (vs base)

Trade-off is honest: persona/tool specialization slightly regressed the general reasoning benchmark, concentrated in the terse short-answer format (MC only −2.5).

Usage

import torch
from transformers import Gemma4ForConditionalGeneration, AutoProcessor, BitsAndBytesConfig
from peft import PeftModel

SKIP=["vision_tower","audio_tower","embed_vision","embed_audio","lm_head"]
bnb=BitsAndBytesConfig(load_in_4bit=True,bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,bnb_4bit_use_double_quant=True,
    llm_int8_skip_modules=SKIP,llm_int8_enable_fp32_cpu_offload=True)
dm={"model.language_model":0,"lm_head":0,"model.vision_tower":"cpu","model.audio_tower":"cpu",
    "model.embed_vision":"cpu","model.embed_audio":"cpu","model.language_model.embed_tokens_per_layer":"cpu"}
m=Gemma4ForConditionalGeneration.from_pretrained("google/gemma-4-E4B-it",quantization_config=bnb,device_map=dm,torch_dtype=torch.bfloat16)
m=PeftModel.from_pretrained(m,"gorkemergune/ayarlicazhocam-gemma-4-e4b")
tok=AutoProcessor.from_pretrained("google/gemma-4-E4B-it").tokenizer
enc=tok.apply_chat_template([{"role":"user","content":"Sen kimsin?"}],
    add_generation_prompt=True,enable_thinking=False,return_tensors="pt",return_dict=True).to(0)
print(tok.decode(m.generate(**enc,max_new_tokens=120)[0][enc["input_ids"].shape[1]:],skip_special_tokens=True))

Training

• QLoRA 4-bit (text backbone only; multimodal towers + elastic per-layer-embeddings kept on CPU).
• LoRA r=16, α=32, dropout=0.05, target = language_model q/k/v/o/gate/up/down.
• 2 epochs, lr 2e-4 OneCycle-cosine, effective batch 16, bnb AdamW8bit, gradient checkpointing.
• Completion-only token-level masking (masks prompt + tool_response spans). Final val loss 0.76. Peak VRAM 9.8 GB.

Note on naming: the original target was gemma-4-12B; E4B was used first to de-risk on a 12 GB GPU. Named -e4b for accuracy. 12B is future work.