Downloads · 30 days
76
7% of all-time downloads
dnabert-minilm
Peltarion/dnabert-minilm
dnabert-minilm is a machine learning model from Peltarion. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
This is a distilled version of DNABERT by using MiniLM technique. It has a BERT architecture with 6 layers and 768 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and meth…
All-time downloads
1.1K
Public
Repo size
378 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin189 MB · 100%
At a glance
- Library
- transformers
- License
- mit
- Model type
- bert
- Access
- Public
- Created
- Jun 9, 2022
- Updated
- Jul 2, 2022
- SHA
- c618bc2e
Model card
From the Hugging Face model README
MiniDNA model
This is a distilled version of DNABERT by using MiniLM technique. It has a BERT architecture with 6 layers and 768 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and methods, please check the original thesis report..
How to Use
The model can be used to fine-tune on a downstream genomic task, e.g. promoter identification.
import torch
from transformers import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained('Peltarion/dnabert-minilm')
More details on how to fine-tune the model, dataset and additional source codes are available on github.com/joanaapa/Distillation-DNABERT-Promoter.
- Library
- transformers
- Type
- bert
- License
- mit
- Created
- Jun 9, 2022
- Updated
- Jul 2, 2022