food-support-copilot-classifier

Food Support Copilot — ticket classifiers

Two scikit-learn logistic-regression classifiers used by the Food Delivery Support Copilot Space.

Inputs

Not raw text. These models consume a 384-dimensional L2-normalized sentence embedding of the customer message, produced by BAAI/bge-small-en-v1.5. Encode with the BGE query prefix, exactly as the app does:

from sentence_transformers import SentenceTransformer
import joblib, numpy as np

encoder = SentenceTransformer("BAAI/bge-small-en-v1.5")
prefix = "Represent this sentence for searching relevant passages: "
vec = encoder.encode([prefix + message], normalize_embeddings=True)[0]

clf = joblib.load("clf_category.joblib")
print(clf.predict(vec.reshape(1, -1))[0])

Metrics

Trained on 10,153 synthetic tickets, 80/20 stratified split, random_state=42.

Intended use and limitations

clf_category.joblib is reliable and is what the app leads with. Its accuracy is high partly because the training data is synthetic and spec-conditioned — the generator was told which category to write about — so expect materially lower numbers on real support tickets.

clf_urgency.joblib does not work and is published for completeness. It scores below the majority-class baseline, because urgency was sampled independently of the text the generator wrote, so the message carries almost no urgency signal. Do not use it to make decisions. See notebook 04.

No sentiment classifier is published: ~84% of the dataset's positive labels contradict their own text, so the app reads sentiment with an LLM instead.

Fitted with scikit-learn 1.9.0. Loading under a different version may fail.