pinhao-gemma4-rag-adapter

Pinhao Gemma 4 RAG Adapter

Public LoRA/PEFT adapter for google/gemma-4-E4B-it.

This is a learning experiment for answer behavior in a local Obsidian RAG workflow. It is not a memory store and does not contain private vault notes.

Intended Use

Use this adapter with local RAG context to encourage:

• short PT-BR answers;
• citations in [S#] format;
• exact refusal when context is insufficient;
• fewer invented citations;
• cleaner answer format for a second-brain assistant.

The knowledge source is still the local RAG system. The adapter only shapes behavior and output format.

Base Model

• Base: google/gemma-4-E4B-it
• Adapter format: LoRA / PEFT
• Task style: chat/text generation
• Serving target: OpenAI-compatible endpoint with output cleanup

Use of this adapter still requires access to, and compliance with, the base Gemma model terms.

Training Data

Training used only synthetic data generated for the lab:

• data/gemma4-rag-contract-train.jsonl: 160 synthetic chat rows
• data/gemma4-rag-contract-eval.jsonl: 40 synthetic holdout rows

The dataset contains fake project names, fake source paths, fake decisions, and fake next steps. It intentionally excludes real Obsidian notes, SQLite indexes, embeddings, personal data, secrets, receipts, CPF, email, phone numbers, and private vault content.

Training Run

• Date: 2026-06-16
• Runner: RunPod PyTorch pod
• GPU: A40 48 GB
• Runtime after setup/cache: 118.7s
• Steps: 20
• Final reported train loss: 1.341
• Stack: torch 2.11.0+cu128, transformers 5.12.1, TRL 1.6.0, PEFT 0.19.1

LoRA targets were limited to model.language_model.* modules to avoid multimodal wrapper incompatibilities.

Evaluation

Holdout eval: 40 synthetic RAG contract rows.

Evaluation Snapshot

Scores are counts over 40 synthetic holdout rows. The adapter changes response behavior, not vault memory.

Interpretation:

• the adapter improved answer-contract behavior from 45% to 92.5%;
• exact refusal improved to 40/40;
• no invented citations were observed in the holdout;
• raw generation can still leak markers such as model, thought, or trailing refusal fragments;
• serving must keep cleanup and token caps in the response path.

Serving Smoke

The adapter was served in an interactive RunPod pod with:

• GET /health
• POST /v1/chat/completions
• local SSH tunnel from 127.0.0.1:18000 to pod 127.0.0.1:8000

End-to-end smoke:

local rag_ask.py -> local retrieval -> SSH tunnel -> RunPod adapter server

Final smoke result:

• local retrieval stayed local;
• only selected prompt context crossed the tunnel;
• response was PT-BR with [S1] and [S2] citations;
• server was stopped after the smoke;
• GPU returned idle.

Example

Question:

o que aconteceu no serving smoke do adapter Pinhao?

Answer:

O serving smoke do adapter Pinhao validou um servidor HTTP mínimo [S1].

*   O servidor respondeu `ok` ao `/health` e teve uma resposta limpa no POST único [S1, S2].
*   O mini-eval HTTP obteve 5/5 no contrato estrito e 5/5 no formato limpo [S1, S2].
*   Não houve envio de vault, SQLite, embeddings ou nota real ao pod [S1].
*   O servidor foi parado após o smoke, e a GPU voltou ao estado idle [S1, S2].

Limitations

• This adapter does not store private knowledge.
• It should not be used without retrieval context for factual answers.
• It still needs serving cleanup and a conservative token cap.
• It was evaluated on a small synthetic holdout, not on broad real-world tasks.
• It is not production-ready without additional evals and monitoring.

Minimal Loading Sketch

from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForMultimodalLM, BitsAndBytesConfig
import torch

base_id = "google/gemma-4-E4B-it"
adapter_id = "brunogonzaga/pinhao-gemma4-rag-adapter"

tokenizer = AutoTokenizer.from_pretrained(adapter_id, use_fast=False)
quantization = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)
base = AutoModelForMultimodalLM.from_pretrained(
    base_id,
    device_map="auto",
    quantization_config=quantization,
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

Privacy

This repository is public and contains adapter artifacts plus documentation only. It was trained on synthetic data and does not include Obsidian vault notes, SQLite indexes, embeddings, secrets, or real personal data. The intended production workflow should still keep the actual vault and retrieval index local/private.