EvSpark

EvSpark — lossless speculative decoding for Evo2

Drafter checkpoints for EvSpark: small distilled drafters that accelerate single-stream Evo2 (StripedHyena2) generation while staying distributionally lossless — greedy output is token-for-token identical to native decoding.

Paper (bioRxiv) · DOI · GitHub · ModelScope

Suite = 48 genomic prompts × 1024 tokens, single stream. Long context: E. coli 262k 1.97–2.43×, B. subtilis 262k 1.84–2.05×. Greedy losslessness: 0 non-tie divergences (48 prompts × 6 ckpts). One checkpoint serves any decode-time draft length γ′ ≤ training γ (exact causal prefix; no retraining). A predictor-guided regulatory-DNA design workflow sees a median 1.57× complete-design speedup over a calibrated batched native baseline (see paper).

Checkpoints

All drafters are single-layer injection (7B: blocks.27; 20B: blocks.20; 40B: blocks.45), d_model=1024, distilled offline from frozen Evo2 hidden states; the target model is never fine-tuned. Each .pt is self-contained (frozen embedding, Markov head, confidence head, metadata). 20B/40B drafters need their targets' official Transformer-Engine path (measured on one H20 96 GB; no official path on Ada).

SHA-256

Quickstart

These files are drafters, not a standalone DNA LM. You need Evo2 + the EvSpark engine (block verify + Hyena/attention state-slice rollback):

git clone https://github.com/dhnihaoya/EvSpark && cd EvSpark   # env setup: see GitHub README
python scripts/download_ckpt.py L27_g12_150M_s1                # HF first, ModelScope fallback
python scripts/demo.py --ckpt L27_g12_150M_s1 --n-tokens 1024

from evspark import EvSpark

with EvSpark.load("L27_g12_150M_s1") as es:
    out = es.generate("ACGTACGT...", n_tokens=1024)   # T=1.0, top_k=4
    g = es.generate(prompt, greedy=True)
    assert g.ids.tolist() == es.generate_native(prompt, greedy=True).ids.tolist()

If huggingface.co is unreachable: HF_ENDPOINT=https://hf-mirror.com, or pull the same files from ModelScope.

Citation

@article{ding2026evspark,
  title   = {EvSpark: Lossless Speculative Decoding for Hybrid DNA Foundation Models},
  author  = {Ding, Hao and Wu, Nannan and Qiu, Tianyi},
  journal = {bioRxiv},
  year    = {2026},
  doi     = {10.64898/2026.09.02.749017},
  url     = {https://www.biorxiv.org/content/10.64898/2026.09.02.749017}
}

License

Checkpoints and code are MIT. Evo2 / Vortex weights and runtime follow their upstream licenses.