mlbn-d192l12-vep-1k

MLBN DNA d192/l12, 1k pretraining checkpoint

This private repository contains the pretrained MLBN DNA checkpoint used to initialize the Caduceus third downstream task (VEP) and the short-to-long transfer ablation.

Intended use

• Initialize the 1k-pretrained rows of the 1k/16k/131k downstream-input-length ablation.
• Keep architecture and checkpoint fixed while changing the downstream input length.
• The scratch control does not use this checkpoint.

Architecture

• d_model: 192
• n_layer: 12
• Vocabulary size: 12
• Context length during pretraining: 1024 (1k)
• Drop path: 0.18
• Training precision: fp32

Files

• last.ckpt: pretrained PyTorch Lightning checkpoint
• mlbn_dna_d192_l12.yaml: model configuration used by the codebase
• model_config.json: model metadata saved with the original run
• training_config.yaml: sanitized training summary without cluster or dataset paths
• SHA256SUMS.txt: checkpoint integrity checksum

Integrity

Expected SHA-256 for last.ckpt:

9f8490ff4cd4c6e3cf9dfd08c1cf22effe223b6ce6a350c7d39d00bf32300f9c

Code

Use this checkpoint with the ablation handoff branch of the project repository:

https://github.com/ZJ4433MR/caduceus-genomic-benchmark/tree/codex/ablation-handoff

This repository intentionally excludes datasets, environment files, and private cluster paths.