distilbert-food-hazard-category-rw

DistilBert Food Hazard Category Classification Model - Random Word Swapping Augmentation

Model Details

Model Description

This model is finetuned on multi-class food hazard-category text classification using random word swapping augmentation and distilbert-base-uncased.

• Developed by: DataScienceWFSR
• Model type: Text Classification
• Language(s) (NLP): English
• Finetuned from model: distilbert/distilbert-base-uncased

Model Sources

• Repository: https://github.com/WFSRDataScience/SemEval2025Task9
• Paper : https://arxiv.org/abs/2504.20703

How to Get Started With the Model

Use the code below to get started with the model in PyTorch.

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from huggingface_hub import hf_hub_download
import pandas as pd

model, category, augmentation = 'distilbert', 'hazard-category', 'rw'

repo_id = f"DataScienceWFSR/{model}-food-{category}-{augmentation}"
lb_path = hf_hub_download(repo_id=repo_id, filename=f"labelencoder_{category}.pkl")
lb =  pd.read_pickle(lb_path)
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
model.eval()

sample = ('Case Number: 039-94 Date Opened: 10/20/1994 Date Closed: 03/06/1995 Recall Class: 1'
        ' Press Release (Y/N): N Domestic Est. Number: 07188 M Name: PREPARED FOODS Imported '
        'Product (Y/N): N Foreign Estab. Number: N/A City: SANTA TERESA State: NM Country: USA'
        ' Product: HAM, SLICED Problem: BACTERIA Description: LISTERIA '
        'Total Pounds Recalled: 3,920 Pounds Recovered: 3,920')

inputs = tokenizer(sample, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)
predictions = outputs.logits.argmax(dim=-1)
predicted_label = lb.inverse_transform(predictions.numpy())[0]
print(f"The predicted label is: {predicted_label}")

Training Details

Training Data

Training and Validation data provided by SemEval-2025 Task 9 organizers : Food Recall Incidents dataset (only English) link

Training Procedure

Training Hyperparameters

• batch_size: 8
• epochs: 3
• lr_scheduler: cosine with Restarts

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data & Metrics

Testing Data

Test data: 997 samples (link)

Metrics

F₁-macro

Results

F₁-macro scores for each model in the official test set utilizing the text field per category and subtasks scores (ST1 and ST2) rounded to 3 decimals. With bold, we indicated the model's specific results.

Technical Specifications

Compute Infrastructure

Hardware

NVIDIA A100 80GB and NVIDIA GeForce RTX 3070 Ti

Software

Citation

BibTeX:

For the original paper:

@inproceedings{brightcookies-semeval2025-task9, 
    title="BrightCookies at {S}em{E}val-2025 Task 9: Exploring Data Augmentation for Food Hazard Classification}, 
    author="Papadopoulou, Foteini and Mutlu, Osman  and Özen, Neris and van der Velden, Bas H. M. and Hendrickx, Iris  and Hürriyetoğlu, Ali",
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul, 
    year = "2025", 
    address = "Vienna, Austria", 
    publisher = "Association for Computational Linguistics", 
} 

For the SemEval2025 Task9:

@inproceedings{semeval2025-task9, 
    title = "{S}em{E}val-2025 Task 9: The Food Hazard Detection Challenge", 
    author = "Randl, Korbinian and Pavlopoulos, John and Henriksson, Aron and Lindgren, Tony and Bakagianni, Juli", 
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul, 
    year = "2025", 
    address = "Vienna, Austria", 
    publisher = "Association for Computational Linguistics", 
} 

Model Card Authors and Contact

Authors: Foteini Papadopoulou, Osman Mutlu, Neris Özen, Bas H.M. van der Velden, Iris Hendrickx, Ali Hürriyetoğlu

Contact: [email protected]