Downloads · 30 days
0
kenlm-pubmed
marianna13/kenlm-pubmed
kenlm-pubmed is a machine learning model from marianna13. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Tells to what extent a text looks like it comes from a scientific paper Trained on Scientific papers dataset
Access
Public
Updated Jun 3, 2023
Repo size
2.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.binary2.3 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jun 3, 2023
- Updated
- Jun 3, 2023
- SHA
- 389d12ee
Model card
From the Hugging Face model README
KenLM trained on PubMed
Tells to what extent a text looks like it comes from a scientific paper
Dataset
Trained on Scientific papers dataset
Installation
!git clone https://huggingface.co/marianna13/kenlm-pubmed
!pip install https://github.com/kpu/kenlm/archive/master.zip -q
Usage
Compute KenLM score:
import kenlm
model = kenlm.LanguageModel('kenlm-pubmed/pubmed.binary')
text = 'The 2019 novel coronavirus (COVID-19) is a newly emerged strain that has never been found in humans before. At present, the laboratory-based reverse transcription-polymerase chain reaction (RT-PCR) is the main method to confirm COVID-19 infection.'
print(model.score(text.lower())) # -84.2962646484375
The less score - the better:
text = 'Kudligi is a panchayat town in Vijayanagara district in the India state of Karnataka'
print(model.score(text.lower())) # -44.775997161865234
Corrupted text:
text = 'Comptition with thool-containiRg'
print(model.score(text.lower())) # -9.675569534301758
- License
- apache-2.0
- Created
- Jun 3, 2023
- Updated
- Jun 3, 2023