dbpedia-hindi-gemma3-4b-lr2e4

markdown

base_model: google/gemma-3-4b-it library_name: peft tags:

• lora
• qlora
• hindi
• relation-extraction
• knowledge-graph
• dbpedia license: gemma language:
• hi

---

DBpedia Hindi — Gemma 3 4B Relational Triple Extraction (lr=2e-4)

A QLoRA fine-tuned adapter for Gemma 3 4B, trained to extract subject-relation-object triples directly from Hindi sentences, as part of the DBpedia Hindi Chapter (Google Summer of Code 2026).

Model Details

• Base model: google/gemma-3-4b-it
• Fine-tuning method: QLoRA (4-bit quantized LoRA)
• Learning rate: 2e-4 (selected checkpoint — outperformed a parallel lr=1e-5 run on every evaluated metric)
• Training framework: PEFT 0.19.1

What It Does

Given a raw Hindi sentence, the model outputs every extractable (subject, relation, object) triple, one per line, in the format:

subject | relation | object

Each relation is also tagged as either a genuine relational fact or property (a non-relational descriptive fragment, e.g. adjective+noun) — property-type triples are excluded from downstream DBpedia matching.

Training Data

• Training set: 39,621 examples (Optimal-trace format), combining an LLM-generated synthetic dataset, a separately generated "noisy" dataset for robustness, and real scraped Hindi Wikipedia sentences
• Validation set: 3,634 examples (high-scoring, ≥9/10, real Wikipedia sentences)

Evaluation Results

Evaluated across three sources: 1,817 Wikipedia sentences, 50 Train sentences, 112 BenchIE sentences (an independent, human-verified benchmark).

Extraction-only (raw triples vs. gold, no DBpedia matching)

Full pipeline (with predicate linking to DBpedia properties)

Compared against an untrained base Gemma 3 4B (same prompt, zero fine-tuning), extraction-only F1: Wikipedia 0.024, Train 0.014, BenchIE 0.016 — fine-tuning improved Wikipedia F1 by roughly 23x and Train F1 by roughly 44x.

Honest note on BenchIE: BenchIE's extraction-only F1 is substantially lower than its full-pipeline F1. Manual inspection found the model consistently trims Hindi postpositions (का/की/में) that BenchIE's gold spans retain, causing exact-match failures despite the core relation being correctly identified in nearly every checked case.

How to Use

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

BASE_MODEL = "google/gemma-3-4b-it"
ADAPTER = "Nitin1211/dbpedia-hindi-gemma3-4b-lr2e4"

tokenizer = AutoTokenizer.from_pretrained(ADAPTER)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, ADAPTER)
model.eval()

Important: generation must stop on either the base eos_token_id or Gemma's <end_of_turn> token — relying on eos_token_id alone causes valid outputs to loop past their real answer.

Part of a Larger Pipeline

This extraction model is the first stage of a full pipeline: extraction → predicate normalization (F2LLM-1.7B) → human review → DBpedia knowledge graph. Full code, documentation, and the normalization model: https://github.com/singhhnitin/neural-extraction-framework/tree/gsoc26h-development/GSoC26_H

License

Inherits Gemma's license terms (see base model). This adapter itself contains no base model weights, only the trained LoRA delta.