bio-posttrain-qwen3-4b-cpt

Bio-posttrain Qwen3-4B CPT

Continued pre-training (CPT) checkpoint from How Post-Training Shapes Biological Reasoning Models.

Model details

• Base model: Qwen/Qwen3-4B
• Stage: CPT (text-only omics corpus)
• Training: lr=1e-5, gradient accumulation=64, final CPT checkpoint

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("mims-harvard/bio-posttrain-qwen3-4b-cpt", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("mims-harvard/bio-posttrain-qwen3-4b-cpt", trust_remote_code=True)

Collection

Part of the Bio-posttrain collection on Hugging Face.

Citation

@article{bio_posttrain_2026,
  title={How Post-Training Shapes Biological Reasoning Models},
  author={...},
  year={2026}
}