Downloads · 30 days
688
4% of all-time downloads
marin-dna/marin-dna-scaling-v0.5-h640-p46M
marin-dna-scaling-v0.5-h640-p46M is a text generation model from marin-dna. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This 45,895,552-parameter nucleotide-level causal language model is the smallest member of the eight-model MarinDNA v0.5 parameter-scaling ladder developed with Marin. This repository contains only the final step-2155…
Downloads · 30 days
688
4% of all-time downloads
All-time downloads
15.7K
Public
Parameters
45.9M
184 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors184 MB · 100%
From the Hugging Face model README
This 45,895,552-parameter nucleotide-level causal language model is the smallest member of the eight-model MarinDNA v0.5 parameter-scaling ladder developed with Marin. This repository contains only the final step-215573 checkpoint from run dna-bolinas-scaling-v0.5-h640-p46M-353f3d, with its tokenizer bundled. It accompanies A 1B standard Transformer rivals Evo 2 40B on variant effect prediction.
| Field | Value |
|---|---|
| Architecture | Qwen3-compatible decoder-only Transformer |
| Parameters | 45,895,552 |
| Layers | 7 |
| Hidden / intermediate size | 640 / 2,560 |
| Attention heads / KV heads | 5 / 5 |
| Context | 256 tokens: one BOS token followed by up to 255 DNA bases |
| Checkpoint | Final checkpoint, step 215573 (after 215,574 zero-indexed training steps) |
| Token exposure | approximately 84.77B nucleotide tokens |
| Stored weight dtype | float32 |
| License | Apache-2.0 |
The canonical source checkpoint is gs://marin-us-east5/checkpoints/dna-bolinas-scaling-v0.5-h640-p46M-353f3d/hf/step-215573; the byte-identical evals_v2 transfer cache is s3://oa-bolinas/snakemake/analysis/evals_v2/results/checkpoints/scaling-v0.5-h640-p46M-step-215573. The commit-pinned training script defines the production ladder.
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "marin-dna/marin-dna-scaling-v0.5-h640-p46M"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)
The bundled tokenizer is case-insensitive and represents one nucleotide per token. Its vocabulary is [PAD]=0, [UNK]=1, [BOS]=2, a=3, c=4, g=5, t=6. Pass raw DNA strings containing A, C, G, and T without spaces. The tokenizer lowercases input and prepends [BOS]; it has no EOS token. Other symbols map to [UNK]. Because BOS occupies one of the 256 positions, inputs are limited to 255 DNA bases.
Every model in the ladder used the same batch size, token budget, optimizer hyperparameters, tokenizer, and three-way training mixture: 73.19% CDS, 20.62% upstream, and 6.19% downstream sequence. Lowercase soft-masked positions received loss weight 0.01, versus 1.0 for uppercase positions. The ladder changes model scale, not the training-data recipe.
Training datasets: CDS, upstream, and downstream. The matched CDS, upstream, and downstream datasets were validation probes, not training data. The exact tokenizer, data, and mixture definitions are commit-pinned here.