Downloads · 30 days
77
4% of all-time downloads
dnabert-minilm-small
Peltarion/dnabert-minilm-small
dnabert-minilm-small is a machine learning model from Peltarion. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
This is a distilled version of DNABERT by using MiniLM technique. It has a BERT architecture with 6 layers and 384 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and meth…
All-time downloads
1.8K
Public
Repo size
159 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin79.3 MB · 100%
At a glance
- Library
- transformers
- License
- mit
- Model type
- bert
- Access
- Public
- Created
- Jun 9, 2022
- Updated
- Jul 2, 2022
- SHA
- 88883e0c
Model card
From the Hugging Face model README
MiniDNA small model
This is a distilled version of DNABERT by using MiniLM technique. It has a BERT architecture with 6 layers and 384 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and methods, please check the original thesis report..
How to Use
The model can be used to fine-tune on a downstream genomic task, e.g. promoter identification.
import torch
from transformers import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained('Peltarion/dnabert-minilm-small')
More details on how to fine-tune the model, dataset and additional source codes are available on github.com/joanaapa/Distillation-DNABERT-Promoter.
- Library
- transformers
- Type
- bert
- License
- mit
- Created
- Jun 9, 2022
- Updated
- Jul 2, 2022