acos-flan-t5-course-evaluation

ACOS FLAN-T5 Course Evaluation Model

This is a fine-tuned google/flan-t5-base model for Aspect-Category-Opinion-Sentiment (ACOS) tuple extraction from university course reviews.

The model is part of the Course ACOS Extraction System, an NLP portfolio project that extracts structured ACOS quadruples from course review text.

Task

Given a course review, the model generates ACOS tuples in the format:

(aspect | category | opinion | sentiment)

## Live Demo

- Streamlit Demo: https://huggingface.co/spaces/lijinzheyy/acos-extraction-system
- Fine-tuned Model: https://huggingface.co/lijinzheyy/acos-flan-t5-course-evaluation

##Example input:

The lecturer explains the concepts clearly but the assignments are too difficult.

##Example output:

(lecturer | faculty comprehensiveness | explains the concepts clearly | positive) ; (assignments | course difficulty | too difficult | negative)

##Model

Base model: google/flan-t5-base
Fine-tuning method: full fine-tuning
Optimizer: AdamW
Epochs: 8
Batch size: 8
fp16: disabled
bf16: disabled
Evaluation

Test set: all-domain OATS-ABSA test set

Model	Precision	Recall	F1
Random Baseline	-	-	0.0000
TF-IDF + kNN Baseline	-	-	0.0579
Old deployed model with post-processing	0.2504	0.2147	0.2312
Final retrained model with post-processing	0.3341	0.2957	0.3137
Deployment

The model is used in a Streamlit demo deployed on Hugging Face Spaces:

Demo: https://huggingface.co/spaces/lijinzheyy/acos-extraction-system
GitHub: https://github.com/JinzheLi-AI/ACOS-Extraction-System
Limitations

The model can still make errors in opinion extraction, category prediction, and sentiment classification. It is designed for course review analysis and may not generalize well to unrelated domains.

##Future Work

Future versions may compare full fine-tuning with LoRA fine-tuning using the same train/dev/test split, post-processing pipeline, and tuple-level exact match evaluation.