Downloads · 30 days
35
1% of all-time downloads
Nhoodie/omni-dna-ici-dc
omni-dna-ici-dc is a text generation model from Nhoodie. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
This model's ICI-DC training underfit near-instantly. The best checkpoint was selected at epoch 0.59 (step 150 of 2530), meaning the model had barely begun learning before early stopping occurred. Despite this, it was…
Downloads · 30 days
35
1% of all-time downloads
All-time downloads
4.4K
Public
Repo size
42 MB
Likes
0
Public
Click a slice to open those files.
.bin42 MB · 99%
From the Hugging Face model README
This model's ICI-DC training underfit near-instantly. The best checkpoint was selected at epoch 0.59 (step 150 of 2530), meaning the model had barely begun learning before early stopping occurred. Despite this, it was used as the basis for SAD fine-tuning (producing Nhoodie/omni-dna-sad-mutation) and its reported metrics were taken at face value.
Subsequent testing revealed that the unmodified base Omni-DNA-20M model achieves 0.951 AUC on discriminative mutation detection with raw DNA prompts — significantly outperforming this fine-tuned variant. The ICI-DC training did not improve and likely degraded the base model's innate mutation understanding.
This model is preserved for historical and reproducibility purposes only.
Interleaved Codon Insertion — Double Consensus (ICI-DC) synthetic pre-training checkpoint.
Omni-DNA-20M fine-tuned on 8,112 synthetic DNA mutation pairs generated by dual-model consensus (Omni-DNA-20M + HyenaDNA tiny-1k). This checkpoint represents the model after learning from synthetic data but before real-data attenuation (SAD).
consensus (2% of gaps), disagreement = contested (98%).| Parameter | Value |
|---|---|
| Base model | zehui127/Omni-DNA-20M (20M params, OLMo causal LM) |
| Training data | 8,112 synthetic mutation pairs |
| Epochs | 10 |
| Learning rate | 5e-5 (cosine schedule) |
| Batch size | 32 effective (16 × 2 grad accum) |
| Precision | fp32 (fp16 causes NaN gradients with Omni) |
| Save format | PyTorch (.bin), no safetensors (tied weights incompatibility) |
| Axis | Metric | Base Omni | ICI-DC (this) | Delta |
|---|---|---|---|---|
| A6: Discriminative | AUC | 0.588 | 0.887 | +0.299 |
| A5: Surprise | Mutation surprise | +0.777 | -0.216 | Model learns to expect mutations |
| A3: Ti/Tv | Predicted ratio | 0.20 | 0.41 | → Biological (0.48) |
| A1: Detection | Recall | 0.511 | 0.518 | +0.007 |
| A2: Logits | Top-3 accuracy | 0.846 | 0.846 | No change |
Key finding: ICI-DC training dramatically improves discriminative ability (AUC +0.30) and teaches the model the biological transition/transversion ratio. The model becomes a strong judge of valid mutation pairs.
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("Nhoodie/omni-dna-ici-dc", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Nhoodie/omni-dna-ici-dc", trust_remote_code=True)
# Score a mutation pair
prompt = "mutate: ATGGCTAGCTGA -> ATAGCTGGCTAA"
logits = model(**tokenizer(prompt, return_tensors="pt")).logits
Git commit: 7be4e73 (branch dev/sad-omni-hyena, private repo)
If using this model, please cite: