GERM-NT-2.5B-multi

Model Card for GERM-NT1

<!-- Provide a quick summary of what the model is/does. -->

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Haozheng Luo, ChengHao Qiu
• License: MIT

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: https://github.com/MAGICS-LAB/GERM
• Paper: https://arxiv.org/abs/2505.00598

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("magicslabnu/GERM-NT-2.5B-multi", trust_remote_code=True)
model = AutoModelForMaskedLM.from_pretrained("magicslabnu/GERM-NT-2.5B-multi", trust_remote_code=True)

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

GLUE

BibTeX:

@misc{luo2025fastlowcostgenomicfoundation,
      title={Fast and Low-Cost Genomic Foundation Models via Outlier Removal}, 
      author={Haozheng Luo and Chenghao Qiu and Maojiang Su and Zhihan Zhou and Zoe Mehta and Guo Ye and Jerry Yao-Chieh Hu and Han Liu},
      year={2025},
      eprint={2505.00598},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2505.00598}, 
}