phi3-agricultural-analyst

Phi-3 Agricultural Analyst (LoRA experiment)

Personal experiment (December 2025). This was my first hands-on LoRA project: a small adapter that teaches Phi-3-mini a structured report format for agricultural questions about 8 US counties. It is a learning artifact, not a production model. Please read What I got wrong before using it.

Summary

What I got wrong (lessons learned)

1. Only 2 of the 7 listed modules were actually adapted. adapter_config.json lists q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj. But Phi-3 uses fused projections (qkv_proj and gate_up_proj), so PEFT matched only o_proj and down_proj. The adapter weights confirm it: 64 tensors for each of those two modules and nothing else (32 layers × 2 modules × LoRA A/B). The correct targets for Phi-3 would have been:

   target_modules = ["qkv_proj", "o_proj", "gate_up_proj", "down_proj"]   # or "all-linear"
   

   What I do now: call model.print_trainable_parameters() and list the LoRA-wrapped module names before every run.
2. The near-zero loss means the model memorized the synthetic template, not that it is very good. Final eval loss was 0.0186 and token accuracy 99.3%. The data was synthetic and highly templated (training loss fell from 1.605 to 0.115 within the first epoch). The 100 validation examples come from the same generator, so the low loss mostly shows the model learned that template.
3. The task evaluation is small and I didn't document it well. "Content Coverage" and "Structure Quality" were scored with my own rubric against the base model. I didn't record the number of items or the exact scoring procedure.
4. The response-time difference isn't a speedup. The fine-tuned model's responses took less time end to end (62.4 s vs 101.9 s). A LoRA adapter doesn't make each token faster. The likely reason is that the fine-tuned model writes shorter, more structured answers. I didn't measure tokens/s or output length separately.

Results (base vs fine-tuned, my rubric)

Training / eval loss by epoch (training loss start → end of epoch):

Training configuration (from training_args.bin and adapter_config.json)

Counties covered: Fresno CA, Kern CA, Lancaster PA, Sioux IA, Yakima WA, Imperial CA, Monterey CA, Deaf Smith TX.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

model_name = "microsoft/Phi-3-mini-4k-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype=torch.bfloat16, device_map="auto",
    trust_remote_code=True, attn_implementation="eager",
)
model = PeftModel.from_pretrained(base_model, "sarathi-balakrishnan/phi3-agricultural-analyst")

prompt = """You are an agricultural analyst. Analyze the following query using the provided data context.

### Query
What are the key investment opportunities in Fresno County agriculture?

### Data Context
County: Fresno, CA
Operators: 5,847
Average Farm Size: 423 acres
Irrigation Coverage: 89%
Revenue per Acre: $2,340

### Analysis"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=500, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

(The data-context numbers above are illustrative example inputs, not verified statistics.)

To merge for deployment: model = model.merge_and_unload().

Output format

The model was trained to produce: a regional summary, key insights, opportunities, risk factors, and a confidence level.

Limitations

• Trained on synthetic data for 8 counties only; expect poor behaviour outside that template.
• Only o_proj / down_proj were adapted, so this is a much weaker adapter than the config implies.
• The evaluation is a small, self-scored rubric. There is no held-out set of real questions.
• Any facts or figures in the output are not verified. This is not agricultural, financial, or investment advice.

What I'd do next

Retrain with the correct fused-module targets, use loss on assistant tokens only, run a rank sweep (r = 8 / 16 / 64), and evaluate on real, human-written questions with a documented rubric.

License

MIT (same as the Phi-3-mini base model).