dbpedia-hindi-gemma3-4b-lr1e5

markdown

base_model: google/gemma-3-4b-it library_name: peft tags:

• lora
• qlora
• hindi
• relation-extraction
• knowledge-graph
• dbpedia license: gemma language:
• hi

---

DBpedia Hindi — Gemma 3 4B Relational Triple Extraction (lr=1e-5)

A QLoRA fine-tuned adapter for Gemma 3 4B, trained to extract subject-relation-object triples from Hindi sentences, as part of the DBpedia Hindi Chapter (Google Summer of Code 2026).

This is the comparison learning-rate run. The lr=2e-4 checkpoint outperformed this one on every evaluated metric and is the recommended checkpoint for actual use. This model is published for completeness and reproducibility of the learning-rate comparison.

Model Details

• Base model: google/gemma-3-4b-it
• Fine-tuning method: QLoRA (4-bit quantized LoRA)
• Learning rate: 1e-5
• Training framework: PEFT 0.19.1

What It Does

Given a raw Hindi sentence, the model outputs every extractable (subject, relation, object) triple, one per line:

subject | relation | object

Each relation is also tagged as either a genuine relational fact or property (a non-relational descriptive fragment).

Training Data

Identical to the lr=2e-4 run — same 39,621-example training set, same 3,634-example validation set — the only variable changed was the learning rate.

Evaluation Results (LLM-as-judge)

How to Use

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

BASE_MODEL = "google/gemma-3-4b-it"
ADAPTER = "Nitin1211/dbpedia-hindi-gemma3-4b-lr1e5"

tokenizer = AutoTokenizer.from_pretrained(ADAPTER)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, ADAPTER)
model.eval()

Important: generation must stop on either the base eos_token_id or Gemma's <end_of_turn> token.

Part of a Larger Pipeline

Full code and documentation: https://github.com/singhhnitin/neural-extraction-framework/tree/gsoc26h-development/GSoC26_H

License

Inherits Gemma's license terms. This adapter contains no base model weights, only the trained LoRA delta.