dairyguard-core1-isolation-forest

DairyGuard Core 1 — Procurement Integrity Isolation Forest

Part of DairyGuard, a fraud detection system for milk procurement in Maharashtra, built for Smart India Hackathon 2026 by team MoneyFollows.py.

Model Description

An IsolationForest (scikit-learn) trained to detect multivariate anomalies in dairy procurement transactions — specifically volume spikes, quality/fat dilution, historical-baseline drift, and temperature anomalies. Duplicate slips and OCR/manual-entry mismatches are handled by separate rule-based and semantic-matching components (not this model).

• Algorithm: Isolation Forest
• n_estimators: 200
• contamination: 0.0127 (calibrated to the true rate of numerically-detectable fraud in the training set, not an arbitrary default)
• Features (species-normalized z-scores):
  • volume_liters_zscore
  • fat_pct_zscore
  • ph_zscore
  • temperature_c_zscore

Features are z-scored within each animal species group (cow/buffalo/goat) before training, since raw fat% and yield differ substantially by species (e.g. buffalo milk fat ~6-7% vs cow ~3.5%) — without this normalization, natural species variation swamps the fraud signal.

Per-animal yield/fat ranges used for simulation are grounded in: Ministry of Fisheries, Animal Husbandry & Dairying, Government of India — Basic Animal Husbandry Statistics 2022 / Integrated Sample Survey (cow yield 3.44–13.49 kg/day depending on breed; buffalo yield 10–16 L/day for Murrah breed; buffalo fat 6–7% vs cow fat 3.5%).

Intended Use

Diagnostic/suggestion-only anomaly scoring for milk procurement transaction review. Not intended for automated remediation or auto-rejection of transactions — flagged records are meant for human review by procurement staff.

Training Data

Trained on a synthetic procurement transaction dataset (~15,340 rows), since no public dataset exists with farmer-level daily milk procurement transactions (this data is operationally sensitive and not published by dairy cooperatives or government bodies). The synthetic generator is calibrated against real data sources:

• Maharashtra Dairy Development Department district-wise monthly procurement volumes (data.gov.in, Open Government Data Platform India)
• 18th Livestock Census, Maharashtra — district-wise cattle/buffalo/goat population counts, used to set realistic species distribution per district
• Veterinary-standard fat%/pH/yield ranges per species

Performance

Evaluated only against numerically-detectable fraud types (volume spikes, quality dilution, baseline deviation, temperature anomalies) — duplicate slips and OCR mismatches are excluded from this evaluation since they don't alter any of the 4 input features and are caught by separate detection layers.

Support: 15,110 normal / 230 fraud rows (final dataset, includes capacity-mismatch as a 5th detectable fraud type).

Why unsupervised, not supervised: We deliberately kept this as an unsupervised Isolation Forest rather than switching to a supervised classifier (which we benchmarked at ~92% precision / 78% recall via 5-fold CV on this synthetic data). The reason: real dairy cooperatives don't have large volumes of confirmed, labeled fraud cases to train on — that's precisely why no public procurement-fraud dataset exists. A supervised model's higher score here mostly reflects it learning our own synthetic injection patterns, which wouldn't generalize to real, previously-unseen fraud tactics. Isolation Forest requires no fraud labels at all, making it the realistic choice for cold-start deployment. A supervised model becomes viable as a phase-2 upgrade once real confirmed fraud cases accumulate post-deployment.

Honest limitation: Fraud detection on subtle, human-mimicked anomalies is a genuinely hard problem — 33% recall at default settings reflects real difficulty separating fraud from natural farmer-to-farmer variation at this data scale, not an implementation flaw. In production, this is one signal among several (paired with rule-based duplicate detection, mass-balance reconciliation, semantic OCR matching, and network-based collusion scoring), not a standalone gate.

How to Use

import joblib
import numpy as np

model = joblib.load("isolation_forest_model.joblib")

# Input must be species-normalized z-scores in this order:
# [volume_liters_zscore, fat_pct_zscore, ph_zscore, temperature_c_zscore]
sample = np.array([[2.1, -1.8, 0.3, 3.2]])

prediction = model.predict(sample)   # -1 = anomaly, 1 = normal
score = model.decision_function(sample)  # lower = more anomalous

Limitations

• Trained on synthetic data — real-world deployment requires recalibration on actual procurement records
• Cannot detect duplicate slips or OCR/text-based mismatches (out of scope for this model by design)
• Contamination rate is tuned to this specific synthetic dataset's fraud injection rate; should be re-tuned for production data
• Diagnostic tool only — flagged predictions require human review, not automated action

Part of DairyGuard Core 1

This model is one of three components in the Procurement Integrity Engine:

1. Isolation Forest (this model) — multivariate volume/quality/temperature anomaly detection
2. Hugging Face Semantic Text Matcher (sentence-transformers/all-MiniLM-L6-v2) — OCR vs manual slip ID matching
3. Mass-Balance Reconciliation — rule-based inflow/outflow variance checks

Plus supporting rule-based checks for duplicate slip detection and NetworkX-based collector collusion risk scoring.