agri-rice-classifier

AMIS Commodity Classifier

This model repository contains artifacts from an AMIS commodity relevance classifier training run. It includes the Transformer model, any configured TF-IDF or sentence-embedding baselines, prediction files, and the training report.

• Dataset: faodl/amis-agri-rice
• Dataset subset: ``
• Dataset revision: main
• Text column: chunk_text
• Label column: label
• Transformer: FacebookAI/xlm-roberta-base
• Generated at: 2026-06-08T17:42:20.320378+00:00

Dataset Summary

Threshold Comparison on Validation Split

Validation metrics document threshold selection and tuning behavior; test metrics remain the primary estimate of out-of-sample performance.

Threshold Comparison on Test Split

Confusion Matrices on Test Split

Rows are true labels and columns are predicted labels.

logistic_tfidf at threshold 0.500

logistic_tfidf at threshold 0.517

xgboost_tfidf at threshold 0.500

xgboost_tfidf at threshold 0.522

embedding-logistic_sentence_embeddings at threshold 0.500

embedding-logistic_sentence_embeddings at threshold 0.617

embedding-svm_sentence_embeddings at threshold 0.500

embedding-svm_sentence_embeddings at threshold 0.496

embedding-lightgbm_sentence_embeddings at threshold 0.500

embedding-lightgbm_sentence_embeddings at threshold 0.037

transformer at threshold 0.500

transformer at threshold 0.966

Validation-Tuned Thresholds

• logistic_tfidf: threshold 0.517 (validation F1 0.882); test F1 change vs 0.5: -0.001.
• xgboost_tfidf: threshold 0.522 (validation F1 0.899); test F1 change vs 0.5: +0.002.
• embedding-logistic_sentence_embeddings: threshold 0.617 (validation F1 0.893); test F1 change vs 0.5: -0.002.
• embedding-svm_sentence_embeddings: threshold 0.496 (validation F1 0.885); test F1 change vs 0.5: +0.000.
• embedding-lightgbm_sentence_embeddings: threshold 0.037 (validation F1 0.892); test F1 change vs 0.5: -0.004.
• transformer: threshold 0.966 (validation F1 0.909); test F1 change vs 0.5: -0.006.

Artifacts

• logistic_tfidf: /content/agri-rice-classifier/baselines/logistic
• xgboost_tfidf: /content/agri-rice-classifier/baselines/xgboost
• embedding-logistic_sentence_embeddings: /content/agri-rice-classifier/baselines/embedding-logistic
• embedding-svm_sentence_embeddings: /content/agri-rice-classifier/baselines/embedding-svm
• embedding-lightgbm_sentence_embeddings: /content/agri-rice-classifier/baselines/embedding-lightgbm
• transformer: /content/agri-rice-classifier/transformer

Inference

Install the runtime dependencies:

pip install transformers torch huggingface_hub pandas joblib scikit-learn xgboost lightgbm

Transformer

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

MODEL_ID = "faodl/agri-rice-classifier"

texts = [
    "Rice export prices increased after new procurement rules were announced.",
    "The finance ministry released its monthly fuel tax bulletin.",
]

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, subfolder="transformer")
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID, subfolder="transformer")
threshold = float(getattr(model.config, "threshold", 0.5))

encoded = tokenizer(
    texts,
    truncation=True,
    padding=True,
    max_length=256,
    return_tensors="pt",
)

with torch.no_grad():
    logits = model(**encoded).logits
    probabilities = torch.softmax(logits, dim=-1)[:, 1].tolist()

for text, probability in zip(texts, probabilities):
    label = model.config.id2label[int(probability >= threshold)]
    print({"text": text, "probability_positive": probability, "label": label})

TF-IDF Baselines

Available baseline names in this run: "logistic", "xgboost".

import json
import joblib
from huggingface_hub import hf_hub_download

MODEL_ID = "faodl/agri-rice-classifier"
BASELINE = "logistic"

texts = [
    "Maize production forecasts were revised after delayed rains.",
    "The central bank published new exchange rate statistics.",
]

model_path = hf_hub_download(
    repo_id=MODEL_ID,
    repo_type="model",
    filename=f"baselines/{BASELINE}/{BASELINE}_tfidf.joblib",
)
report_path = hf_hub_download(
    repo_id=MODEL_ID,
    repo_type="model",
    filename="report.json",
)

pipeline = joblib.load(model_path)
with open(report_path, encoding="utf-8") as handle:
    report = json.load(handle)

threshold = next(
    result["validation_best_threshold"]["threshold"]
    for result in report["results"]
    if result["model_type"] == f"{BASELINE}_tfidf"
)

probabilities = pipeline.predict_proba(texts)[:, 1]
for text, probability in zip(texts, probabilities):
    label = "RELEVANT" if probability >= threshold else "NOT_RELEVANT"
    print({"text": text, "probability_positive": float(probability), "label": label})

Sentence-Embedding Baselines

Available embedding baseline names in this run: "embedding-logistic", "embedding-svm", "embedding-lightgbm".

import joblib
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModel, AutoTokenizer

MODEL_ID = "faodl/agri-rice-classifier"
BASELINE = "embedding-logistic"

texts = [
    "Wheat export inspections rose as demand from importers increased.",
    "The sports ministry announced a new stadium renovation plan.",
]

model_path = hf_hub_download(
    repo_id=MODEL_ID,
    repo_type="model",
    filename=f"baselines/{BASELINE}/{BASELINE}.joblib",
)
artifact = joblib.load(model_path)
tokenizer = AutoTokenizer.from_pretrained(artifact["embedding_model_name"])
encoder = AutoModel.from_pretrained(artifact["embedding_model_name"])
encoder.eval()

encoded_batches = []
batch_size = artifact.get("embedding_batch_size", 64)
for start in range(0, len(texts), batch_size):
    batch_texts = texts[start : start + batch_size]
    inputs = tokenizer(
        batch_texts,
        padding=True,
        truncation=True,
        max_length=artifact.get("embedding_max_length", 256),
        return_tensors="pt",
    )
    with torch.no_grad():
        outputs = encoder(**inputs)
    token_embeddings = outputs.last_hidden_state
    attention_mask = inputs["attention_mask"].unsqueeze(-1).to(token_embeddings.dtype)
    embeddings = (token_embeddings * attention_mask).sum(dim=1)
    embeddings = embeddings / attention_mask.sum(dim=1).clamp(min=1e-9)
    if artifact.get("normalize_embeddings", True):
        embeddings = torch.nn.functional.normalize(embeddings, p=2, dim=1)
    encoded_batches.append(embeddings)
embeddings = torch.cat(encoded_batches).numpy()
probabilities = artifact["classifier"].predict_proba(embeddings)[:, 1]
threshold = artifact["validation_best_threshold"]["threshold"]

for text, probability in zip(texts, probabilities):
    label = "RELEVANT" if probability >= threshold else "NOT_RELEVANT"
    print({"text": text, "probability_positive": float(probability), "label": label})

Files

• REPORT.md: Markdown report for this training run.
• report.json: Machine-readable report containing metrics and thresholds.
• transformer/: Fine-tuned Transformer artifacts, when Transformer training is enabled.
• baselines/: TF-IDF and sentence-embedding baseline artifacts, when baseline training is enabled.
• */validation_predictions.csv and */test_predictions.csv: Split-level predictions.