Do_an_group_BiLSTM_CRF

BiLSTM-CRF-FoodNER: Deep Sequence Labeling Baseline for Vietnamese Food Orders

Table of contents

1. Introduction
2. Dataset Overview
3. Entity Label Space
4. Empirical Evaluation
5. Architecture Characteristics
6. Using BiLSTM-CRF-FoodNER with PyTorch
7. Authors & Citation

---

<a name="introduction"></a> 1. Introduction

CS221DoAn/Do_an_group_BiLSTM_CRF is a deep learning model based on the Bidirectional LSTM + Conditional Random Field (BiLSTM-CRF) architecture. It is specifically trained for Named Entity Recognition (NER) on domain-specific Vietnamese unstructured text: Online Food Delivery Orders and Messages.

In our research project for the CS221.Q21 course (Natural Language Processing), this model serves as a deep-learning sequence baseline. It was developed to overcome the limitations of traditional CRF (which relies heavily on manual feature engineering) by automating non-linear feature extraction through recurrent neural networks.

<a name="dataset"></a> 2. Dataset Overview

The model was trained on a custom, manually annotated dataset consisting of 2,325 real-world food ordering messages extracted from Facebook comments (specifically from Sơn Nguyễn vegetarian restaurant).

• Train Set: 1,860 samples (80%)
• Validation Set: 232 samples (10%)
• Test Set: 233 samples (10%)

<a name="labels"></a> 3. Entity Label Space

The model utilizes the standard BIO (Begin - Inside - Outside) tagging scheme across 15 distinct entity labels, mapping to 7 core information classes:

<a name="evaluation"></a> 4. Empirical Evaluation

The model was evaluated on an unseen Test Set using strict Entity-level F1-Scores via the seqeval framework.

• BiLSTM-CRF Macro F1-Score: 0.9771
• Comparison: While the BiLSTM-CRF automates feature extraction, it achieved the lowest Macro F1-score (0.9771) among our tested baselines (CRF: 0.9889, mBERT: 0.9880, PhoBERT: 0.9913).

<a name="characteristics"></a> 5. Architecture Characteristics

The data flows through 3 main layers in this architecture:

1. Embedding Layer: Converts numerical vocabulary identifiers into continuous mathematical vectors.
2. BiLSTM Layer: Processes the sequence in both directions (left-to-right and right-to-left). At each time step $t$, the hidden state is a concatenation of both directions: $h_{t}=[\vec{h}{t};\vec{h{t}}]$.
3. CRF Layer: Receives emission scores from the BiLSTM and applies a transition matrix to find the most valid sequence of tags via Viterbi Decoding.

Limitation: Our analysis showed that BiLSTM-CRF suffers from Information Decay when dealing with long food orders where related entities are far apart (e.g., QUANTITY at the beginning and FOOD at the end). The Self-Attention mechanism in Transformer models (like PhoBERT) resolves this limitation efficiently.

<a name="usage"></a> 6. Using BiLSTM-CRF-FoodNER with PyTorch

Because this is a custom PyTorch model, you must define the model architecture class in your local environment before loading the .pth weights.

Installation

pip install torch huggingface_hub

Example usage (Inference Pipeline)

⚠️ IMPORTANT: You MUST replace the BiLSTM_CRF class placeholder below with the exact PyTorch class definition you used during the training phase.

import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download

# 1. DEFINE YOUR MODEL ARCHITECTURE (MUST MATCH TRAINING CODE)
class BiLSTM_CRF(nn.Module):
    def __init__(self, vocab_size, tagset_size, embedding_dim, hidden_dim):
        super(BiLSTM_CRF, self).__init__()
        # --- PASTE YOUR ACTUAL PYTORCH INIT CODE HERE ---
        pass
        
    def forward(self, sentence):
        # --- PASTE YOUR ACTUAL FORWARD PASS HERE ---
        pass
        
    def decode(self, sentence):
        # --- PASTE YOUR VITERBI DECODING HERE ---
        pass

# Initialize the model with your original hyperparameters
# model = BiLSTM_CRF(vocab_size=..., tagset_size=15, embedding_dim=..., hidden_dim=...)

# 2. DOWNLOAD AND LOAD WEIGHTS
REPO_ID = "CS221DoAn/Do_an_group_BiLSTM_CRF"
FILENAME = "bilstm_crf.pth"

print("Downloading BiLSTM-CRF weights...")
model_path = hf_hub_download(repo_id=REPO_ID, filename=FILENAME)

# Load state dict
# model.load_state_dict(torch.load(model_path, map_location=torch.device('cpu')))
# model.eval()

print("Model loaded successfully! Ready for inference.")

# 3. PREDICT (Replace with your actual text processing pipeline)
def predict_food_order(raw_text):
    print(f"\nInput: {raw_text}")
    # 1. Preprocess & Tokenize text
    # 2. Convert to Tensor
    # 3. Pass through model.decode()
    # 4. Map ID to Label
    pass

7. Authors & Citation

This project was developed for the Natural Language Processing (CS221.Q21) course at the University of Information Technology (UIT) - VNU-HCM.

• Students: Võ Thành Lộc (24520989), Nguyễn Anh Nguyên (24521185)

• Instructor: Ph.D. Nguyễn Trọng Chỉnh

• Date: July 2026

If you use this model in your academic research or projects, please cite our project:

@misc{cs221_food_order_ner_bilstm_crf,
  author = {Vo Thanh Loc and Nguyen Anh Nguyen},
  title = {Food Order Extraction: BiLSTM-CRF Baseline for Vietnamese NER},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = https://huggingface.co/CS221DoAn/Do_an_group_BiLSTM_CRF
}