Downloads · 30 days
71
6% of all-time downloads
dnabert-minilm-mini
Peltarion/dnabert-minilm-mini
dnabert-minilm-mini is a machine learning model from Peltarion. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
This is a distilled version of DNABERT by using MiniLM technique. It has a BERT architecture with 3 layers and 384 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and meth…
All-time downloads
1.2K
Public
Repo size
87.6 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin43.8 MB · 100%
At a glance
- Library
- transformers
- License
- mit
- Model type
- bert
- Access
- Public
- Created
- Jun 9, 2022
- Updated
- Jul 2, 2022
- SHA
- 22478613
Model card
From the Hugging Face model README
MiniDNA mini model
This is a distilled version of DNABERT by using MiniLM technique. It has a BERT architecture with 3 layers and 384 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and methods, please check the original thesis report..
How to Use
The model can be used to fine-tune on a downstream genomic task, e.g. promoter identification.
import torch
from transformers import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained('Peltarion/dnabert-minilm-mini')
More details on how to fine-tune the model, dataset and additional source codes are available on github.com/joanaapa/Distillation-DNABERT-Promoter.
- Library
- transformers
- Type
- bert
- License
- mit
- Created
- Jun 9, 2022
- Updated
- Jul 2, 2022