TropicBERT

TropicBERT: Tropical Fruit Genomics Foundation Models

---

🧬 1. Model Overview

TropicBERT is the first genomic foundation model series specifically designed for tropical fruit crop genome sequences. We pre-trained BERT models using the MLM (Masked Language Modeling) objective on datasets comprising 13 different species combinations, releasing a total of 13 pre-trained model variants covering 1, 5, and 10 species.

The models are developed based on TropicBERT-LLMs_One_stop_tutorial, a reproducible pipeline that successfully transfers the "pre-training and fine-tuning" paradigm from NLP to genomics, significantly lowering the barrier for developing plant genomic large language models.

🔗 Related Resources：

• GitHub Full Tutorial: TropicBERT-LLMs_One_stop_tutorial
  • Includes data preprocessing, pre-training, and fine-tuning scripts
  • Provides sample datasets in FASTA/CSV format
• The original genomic data is sourced from： tropicalfruit-omics database
• Published Paper: [ Paper ]
  • More detailed model description and training details

---

📖 2. Model Variants

We provide 13 model versions trained on different species combinations to meet various research needs：

---

🚀 3. Quick Usage

The following code demonstrates how to load the model and extract DNA sequence embeddings.

from transformers import AutoTokenizer, AutoModel
import torch

model_name = "yang0104/TropicBERT"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

seq = "ATGCGTACGTTAGCCTA..."

inputs = tokenizer(seq, return_tensors="pt")
outputs = model(**inputs)

embeddings = outputs.last_hidden_state
print(embeddings.shape)

📞 4. Contact & Citation

If you encounter any issues or wish to discuss collaboration, please contact us via:

• 🛠 Submit Issue: GitHub Issues
• 💬 Community Discussion: HuggingFace Discussions
• 📧 Email: [email protected], [email protected]

Citation: If you use TropicBERT in your research, please cite the following paper:

paper