Downloads · 30 days
2.7K
0% of all-time downloads
unieval-fact
MingZhong/unieval-fact
unieval-fact is a machine learning model from MingZhong. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Pre-trained evaluator in EMNLP 2022 paper
All-time downloads
724K
Public
Repo size
12.5 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.pt6.3 GB · 67%
At a glance
- Library
- transformers
- Model type
- t5
- Access
- Public
- Created
- Oct 12, 2022
- Updated
- Oct 14, 2022
- SHA
- 68e455e9
Model card
From the Hugging Face model README
Pre-trained evaluator in EMNLP 2022 paper
Towards a Unified Multi-Dimensional Evaluator for Text Generation
Introduction
Multi-dimensional evaluation is the dominant paradigm for human evaluation in Natural Language Generation (NLG), i.e., evaluating the generated text from multiple explainable dimensions, such as coherence and fluency.
However, automatic evaluation in NLG is still dominated by similarity-based metrics (e.g., ROUGE, BLEU), but they are not sufficient to portray the difference between the advanced generation models.
Therefore, we propose UniEval to bridge this gap so that a more comprehensive and fine-grained evaluation of NLG systems can be achieved.
Pre-trained Evaluator
unieval-fact is the pre-trained evaluator for the factual consistency detection task. It can evaluate the model output and predict a consistency score.
Usage
Please refer to our GitHub repository.
- Library
- transformers
- Type
- t5
- Created
- Oct 12, 2022
- Updated
- Oct 14, 2022