Downloads · 30 days
19K
5% of all-time downloads
refutation_detector_distilbert
garak-llm/refutation_detector_distilbert
refutation_detector_distilbert is a machine learning model from garak-llm. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This classifier is trained on LLM responses designed to capture a model refuting a false claim.
All-time downloads
357K
Public
Parameters
67M
268 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors268 MB · 100%
At a glance
- License
- apache-2.0
- Model type
- distilbert
- Access
- Public
- Created
- Aug 27, 2024
- Updated
- Aug 27, 2024
- SHA
- 906ac60c
Model card
From the Hugging Face model README
This classifier is trained on LLM responses designed to capture a model refuting a false claim.
They're prompt results of OpenAI gpt-3.5-turbo run on June 1 2023. Prompts are constructed by prepending "Explain why" to claims made in the True-False dataset provided alongside The Internal State of an LLM Knows When its Lying. Only the first sentence of the response is included (from nltk.sent_tokenize). The original labels are used, where 0 corresponds to a false claim. That is, the 0 labels should be almost all refutations. Spurious and missing refutations were removed by hand, corresponding to about 10% of the data. This were caused by either an incorrect model assertion, or errors in the source data.
- Type
- distilbert
- License
- apache-2.0
- Languages
- en
- Created
- Aug 27, 2024
- Updated
- Aug 27, 2024