Downloads · 30 days
15
2% of all-time downloads
ComVE-gpt2-medium
aliosm/ComVE-gpt2-medium
ComVE-gpt2-medium is a feature extraction model from aliosm. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as mit.
Finetuned model on Commonsense Validation and Explanation (ComVE) dataset introduced in SemEval2020 Task4 using a causal language modeling (CLM) objective. The model is able to generate a reason why a given natural la…
All-time downloads
946
Public
Repo size
4.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1.4 GB · 50%
At a glance
- Task
- Feature Extraction
- Library
- transformers
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- May 21, 2021
- SHA
- f7591769
Try a prompt
Datasets
Model card
From the Hugging Face model README
ComVE-gpt2-medium
Model description
Finetuned model on Commonsense Validation and Explanation (ComVE) dataset introduced in SemEval2020 Task4 using a causal language modeling (CLM) objective. The model is able to generate a reason why a given natural language statement is against commonsense.
Intended uses & limitations
You can use the raw model for text generation to generate reasons why natural language statements are against commonsense.
How to use
You can use this model directly to generate reasons why the given statement is against commonsense using generate.sh script.
Note: make sure that you are using version 2.4.1 of transformers package. Newer versions has some issue in text generation and the model repeats the last token generated again and again.
Limitations and bias
The model biased to negate the entered sentence usually instead of producing a factual reason.
Training data
The model is initialized from the gpt2-medium model and finetuned using ComVE dataset which contains 10K against commonsense sentences, each of them is paired with three reference reasons.
Training procedure
Each natural language statement that against commonsense is concatenated with its reference reason with <|continue|> as a separator, then the model finetuned using CLM objective.
The model trained on Nvidia Tesla P100 GPU from Google Colab platform with 5e-5 learning rate, 5 epochs, 128 maximum sequence length and 64 batch size.
Eval results
The model achieved fifth place with 16.7153/16.1187 BLEU scores and third place with 1.94 Human Evaluation score on SemEval2020 Task4: Commonsense Validation and Explanation development and testing dataset.
These are some examples generated by the model:
| Against Commonsense Statement | Generated Reason |
|---|---|
| Chicken can swim in water. | Chicken can't swim. |
| shoes can fly | Shoes are not able to fly. |
| Chocolate can be used to make a coffee pot | Chocolate is not used to make coffee pots. |
| you can also buy tickets online with an identity card | You can't buy tickets with an identity card. |
| a ball is square and can roll | A ball is round and cannot roll. |
| You can use detergent to dye your hair. | Detergent is used to wash clothes. |
| you can eat mercury | mercury is poisonous |
| A gardener can follow a suspect | gardener is not a police officer |
| cars can float in the ocean just like a boat | Cars are too heavy to float in the ocean. |
| I am going to work so I can lose money. | Working is not a way to lose money. |
BibTeX entry and citation info
@article{fadel2020justers,
title={JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models Against Commonsense Validation and Explanation},
author={Fadel, Ali and Al-Ayyoub, Mahmoud and Cambria, Erik},
year={2020}
}
- Task
- Feature Extraction
- Library
- transformers
- Type
- gpt2
- License
- mit
- Languages
- en
- Created
- Mar 2, 2022
- Updated
- May 21, 2021