Downloads · 30 days
81
5% of all-time downloads
dnabert-distilbert
Peltarion/dnabert-distilbert
dnabert-distilbert is a fill-mask model from Peltarion. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as mit.
This is a distilled version of DNABERT by using DistilBERT technique. It has a BERT architecture with 6 layers and 768 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and…
All-time downloads
1.7K
Public
Repo size
373 MB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.bin187 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- mit
- Model type
- distilbert
- Access
- Public
- Created
- Jun 9, 2022
- Updated
- Jul 2, 2022
- SHA
- eb657558
Try a prompt
Model card
From the Hugging Face model README
DistilDNA model
This is a distilled version of DNABERT by using DistilBERT technique. It has a BERT architecture with 6 layers and 768 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and methods, please check the original thesis report.
How to Use
The model can be used to fine-tune on a downstream genomic task, e.g. promoter identification.
import torch
from transformers import DistilBertForSequenceClassification
model = DistilBertForSequenceClassification.from_pretrained('Peltarion/dnabert-distilbert')
More details on how to fine-tune the model, dataset and additional source codes are available on github.com/joanaapa/Distillation-DNABERT-Promoter.
- Task
- Fill-Mask
- Library
- transformers
- Type
- distilbert
- License
- mit
- Created
- Jun 9, 2022
- Updated
- Jul 2, 2022