adtc-agri-sft-gemma2-2b

adtc-agri-sft-gemma2-2b

This model is a fine-tune of google/gemma-2-2b-it, adapted via QLoRA (4-bit base + LoRA adapters, merged back into full weights) for agricultural advisory Q&A aimed at smallholder farmers and extension officers in Nigeria.

Part of the ADTC 2026 project. This is a baseline SFT run — direct supervised fine-tuning with no distillation — used to establish how much headroom a distillation stage could still add over the base model.

Model description

• Base model: google/gemma-2-2b-it

• Fine-tuning method: QLoRA (4-bit NF4 quantized base, bf16 compute dtype, double quantization), adapters merged into the base weights for the published checkpoint

• LoRA config: r=16, alpha=32, dropout=0.05, target modules: q_proj, down_proj, up_proj, gate_proj, k_proj, o_proj, v_proj

• Task: single-turn instruction following — given a farming scenario or question, produce a concise assessment, a practical recommendation, and (if relevant) a note on when to escalate to in-person expert help

• System prompt used at train/inference time: Gemma-2's chat template does not support a system role, so the prompt below is prepended into the first user turn rather than sent as a separate message:

  You are an offline agricultural advisory assistant for smallholder farmers and extension officers in Nigeria. Given the situation described, provide a clear assessment, a practical recommendation, and, where relevant, a note on when to seek in-person expert help. Be concise and avoid unnecessary hedging.

Training data

Combined and deduplicated (on normalized question text) from:

• kaggle_csv: 36,489 examples

• synthetic_market: 593 examples

• synthetic_fertilizer: 598 examples

• Total training examples: 35,796

• Held-out eval examples: 1,884 (5% split)

• Primary source: the Kaggle agriculture_qa_final_cleaned.csv dataset (query/response columns), supplemented with synthetically generated fertilizer-dosage and market/storage economics word problems to cover numeric-reasoning cases underrepresented in the base dataset.

Training procedure

Hyperparameters

Training loss

Intended use & limitations

• Intended for offline agricultural advisory use cases (smallholder farmers, extension officers) in a Nigerian context; not validated for general-purpose assistant use or for regions/crops outside its training distribution.
• This is a baseline checkpoint for internal comparison against the un-tuned base model and, later, a distilled variant — treat outputs as advisory, not authoritative agronomic or financial guidance, and always recommend in-person expert follow-up for high-stakes decisions.
• Evaluated informally via qualitative spot-checks on held-out prompts; formal accuracy benchmarking (ARC-Easy / domain-quality via the ADTC profiler) is tracked separately from this card.

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "EYEDOL/adtc-agri-sft-gemma2-2b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

# Gemma-2 has no "system" role — fold the system prompt into the user turn
messages = [
    {"role": "user", "content": "You are an offline agricultural advisory assistant for smallholder farmers and extension officers in Nigeria. Given the situation described, provide a clear assessment, a practical recommendation, and, where relevant, a note on when to seek in-person expert help. Be concise and avoid unnecessary hedging.

Your farming question here"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

---

Card generated automatically at training time — 2026-09-14 11:18 UTC.