gemma4-e2b-repomind-qlora-synth-v2-pilot

Gemma4 E2B RepoMind QLoRA Synth v2 Pilot

This repository contains a PEFT QLoRA adapter for google/gemma-4-E2B-it, trained for RepoMind call-chain analysis experiments.

It does not include the Gemma base model weights. Users must have access to google/gemma-4-E2B-it and comply with the base model license/terms.

Intended Use

The adapter is intended for controlled research and evaluation of repo-local symbol-level call edge extraction. It was trained on synthetic call-chain samples only. AstrBot and Scrapy real evaluation cases were not mixed into the training set.

This is a pilot adapter, not a final production model. Synthetic dev loss improved strongly, but real-case evaluation did not show a net gain over the previous adapter.

Training Summary

• Base model: google/gemma-4-E2B-it
• Dataset: full-synthetic-augmented-v2-20260621
• Dataset SHA256: 09601acfabebab623f543d76366b186f2da0bbc9143a69ffaff00e476802781a
• Train/dev samples: 400 / 100
• Max sequence length: 512
• Steps: 100
• LoRA targets: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
• Excluded modules: regex:.*(vision_tower|audio_tower).*
• Adapter SHA256: 8ef422a6eeb142d657da41deeb41abae9e8ca3e7f11f5fb1605b0b6af845b2c7

Synthetic dev loss:

Overfit monitor assessment: no_overfit_signal_eval_loss_decreased.

Real-Case Evaluation

Four real repository cases were evaluated with the same Oracle Context prompt:

Conclusion: the v2 adapter improved synthetic dev loss but did not produce a net improvement on the 4-case real-repo smoke test. Use it as an experimental checkpoint.

Loading Example

import torch
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

adapter_id = "Shirodora233/gemma4-e2b-repomind-qlora-synth-v2-pilot"

tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
model = AutoPeftModelForCausalLM.from_pretrained(
    adapter_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "system",
        "content": (
            "Return repo-local symbol-level call edges as strict JSON. "
            "Use fully qualified symbols, include file/line/evidence for every call edge."
        ),
    },
    {
        "role": "user",
        "content": "{\"case_id\":\"example\",\"question\":\"Find repo-local call edges.\"}",
    },
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=768,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )

generated = output_ids[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(generated, skip_special_tokens=True))

Included Files

• adapter_model.safetensors
• adapter_config.json
• tokenizer.json
• tokenizer_config.json
• chat_template.jinja
• metadata/training_summary.json
• metadata/overfit_monitor.json
• metadata/run_config.json
• metadata/environment_snapshot.json
• metadata/freeze-manifest.json
• metadata/validation-summary.json
• reports/finetune-gemma4-e2b-qlora-frozen-synth-v2-100step-20260621.md
• reports/finetune-gemma4-e2b-realcase-v1-v2-comparison-20260621.md