deeptaxa

DeepTaxa: Hierarchical 16S rRNA Taxonomy Classification

DeepTaxa is a deep learning model for hierarchical taxonomy classification of 16S rRNA gene sequences. The architecture couples a convolutional branch, which captures local k-mer motifs, with a BERT-style transformer, which captures long-range context. Both branches operate over tokens produced by the DNABERT-2 byte-pair encoder. Predictions are generated jointly for all seven standard taxonomic ranks: domain, phylum, class, order, family, genus, and species.

Three model families are released here, one trained on full-length 16S sequences, one trained on V3-V4 amplicons, and one trained on the shorter V4 amplicon. As of the v2 release each family is provided as five checkpoints trained under identical settings with different random seeds (42, 123, 456, 789, and 1011), enabling ensembling and cross-seed uncertainty estimates, alongside a single default checkpoint for users who need only one model.

What is new in v2

• Five seeds per region. Every family now ships all five seeds (42, 123, 456, 789, 1011) as deeptaxa-<region>-v2-seed<N>.pt, so the released set is directly reproducible and suitable for ensembling. A default single model, deeptaxa-<region>-v2.pt, is provided as a copy of the seed 42 checkpoint for backward-compatible single-model use.
• Self-describing checkpoints. Each checkpoint records its own seed in a seed field, so an ensemble member can be identified from the file contents alone, independent of the filename.
• Reported cross-seed statistics. All three families now report mean and standard deviation across five seeds at every rank (previously only the full-length family reported cross-seed statistics, over three seeds, and the V4 family was single-seed).
• Same recipe as v1. The architecture, hyperparameters, training data, and in-silico amplicon extraction are identical to v1. v2 is a fresh five-seed retraining on the current codebase, not a methodological change. The v1 checkpoints remain available in this repository.

Checkpoint selection

For maximum accuracy and calibrated uncertainty, average the softmax probabilities of all five seed checkpoints for the chosen region (see Ensembling).

Released checkpoints

Species-level test-set performance, reported as the mean and standard deviation across the five seeds:

Standard deviations are given in percentage points. The cross-seed standard deviation is at most 0.09 percentage points of species F1 in every family, indicating high reproducibility. The v2 means match the v1 single-seed and three-seed numbers to within the cross-seed spread.

All checkpoints are inference-only. Optimizer, scheduler, scaler, and RNG state, along with the training/validation split, have been removed to reduce file size; resuming training from these checkpoints is not supported.

Seeds and ensembling

Each region provides five seed checkpoints and one default. The five differ only in their random seed; they share architecture, hyperparameters, and training data.

A simple and effective ensemble averages the per-rank softmax probabilities across the five seeds and then takes the argmax at each rank. Because the seeds are independent training runs of the same configuration, this reduces variance and typically improves both accuracy and calibration relative to any single seed. When only one model is needed, use the default (seed 42) checkpoint.

Architecture

The full-length, V3-V4, and V4 families share the same compact HybridCNNBERT configuration.

Test-set performance

Each family was evaluated on its respective held-out Greengenes2 2024.09 test split. Numbers below are the five-seed mean at each rank.

Training configuration

Usage

Download

# Default single models (seed 42) for each region
wget https://huggingface.co/systems-genomics-lab/deeptaxa/resolve/main/deeptaxa-full-length-v2.pt
wget https://huggingface.co/systems-genomics-lab/deeptaxa/resolve/main/deeptaxa-v3v4-v2.pt
wget https://huggingface.co/systems-genomics-lab/deeptaxa/resolve/main/deeptaxa-v4-v2.pt

# All five seeds for a region (example: full-length), for ensembling
for s in 42 123 456 789 1011; do
  wget https://huggingface.co/systems-genomics-lab/deeptaxa/resolve/main/deeptaxa-full-length-v2-seed${s}.pt
done

# Or clone the full repository
git clone https://huggingface.co/systems-genomics-lab/deeptaxa

After downloading, verify each file against its SHA-256 checksum. The SHA-256 sums for all v2 files are listed in SHA256SUMS; the default models are:

sha256sum --check SHA256SUMS

Python API with huggingface_hub:

from huggingface_hub import hf_hub_download

# Default single model (seed 42)
full_length_ckpt = hf_hub_download(
    repo_id="systems-genomics-lab/deeptaxa",
    filename="deeptaxa-full-length-v2.pt",
)

# All five seeds for ensembling
seed_ckpts = [
    hf_hub_download(
        repo_id="systems-genomics-lab/deeptaxa",
        filename=f"deeptaxa-full-length-v2-seed{s}.pt",
    )
    for s in (42, 123, 456, 789, 1011)
]

Install DeepTaxa and run predictions

pip install git+https://github.com/systems-genomics-lab/deeptaxa.git

# Full-length sequences (Sanger, PacBio HiFi, Oxford Nanopore)
deeptaxa predict \
  --fasta-file your_full_length_16s.fna.gz \
  --checkpoint deeptaxa-full-length-v2.pt \
  --output-dir predictions/

# V3-V4 amplicons (Illumina, already demultiplexed and primer-trimmed)
deeptaxa predict \
  --fasta-file your_v3v4_amplicons.fna.gz \
  --checkpoint deeptaxa-v3v4-v2.pt \
  --output-dir predictions/

# V4 amplicons (Illumina, already demultiplexed and primer-trimmed)
deeptaxa predict \
  --fasta-file your_v4_amplicons.fna.gz \
  --checkpoint deeptaxa-v4-v2.pt \
  --output-dir predictions/

Input preparation for amplicon checkpoints: the input FASTA file should contain region-matched sequences that have already been demultiplexed and primer-trimmed by an upstream tool such as DADA2, cutadapt, or QIIME2. The V3-V4 and V4 checkpoints were trained on in-silico primer extractions (341F/805R and 515F/806R respectively), which approximate merged paired-end amplicons. Paired-end reads should therefore be merged into consensus amplicons prior to prediction, or the forward read alone may be provided.

Full usage documentation and analysis notebooks are available in the GitHub repository.

Limitations

Limitations that apply to all checkpoints:

• Approximately 44.8% of Greengenes2 species have only a single training example, which limits reliable prediction for those classes.
• The label space corresponds to Greengenes2 2024.09. Predictions are produced against the exact Greengenes2 hierarchy, and species absent from the training data cannot be predicted. Adapting the model to a different reference database, such as SILVA or GTDB, would require retraining.
• A GPU is strongly recommended; CPU inference is impractical for large datasets.

Limitations specific to the full-length family:

• Best performance is obtained on sequences of at least 1,200 bp; shorter amplicons should be classified using the V3-V4 or V4 family.
• Species-level accuracy plateaus near 93%.

Limitations specific to the V3-V4 family:

• Species-level accuracy plateaus near 87.5%. The approximately 420 bp V3-V4 region carries less taxonomic information than the full 16S gene.
• The label space contains 8,347 species. Those for which no V3-V4 amplicon could be extracted during training are absent and cannot be predicted.
• Primer specificity: the models were trained on 341F/805R extractions. Sequences amplified with other V3-V4 primers, such as 357F or 338F, or with substantially different region boundaries may yield degraded predictions.

Limitations specific to the V4 family:

• Species-level accuracy is approximately 82.8%. The approximately 253 bp V4 region carries less taxonomic information than V3-V4 or the full 16S gene, so species-level calls should be read together with their confidence scores.
• The label space contains 16,909 species (the same as the full-length family), retained because V4 amplicons were extracted at 99.0% yield. Species for which no V4 amplicon could be extracted during training are absent and cannot be predicted.
• Primer specificity: the models were trained on 515F/806R extractions. Sequences amplified with other V4 primers or with substantially different region boundaries may yield degraded predictions.

Citation

If DeepTaxa contributes to your research, please cite our paper in Bioinformatics Advances: https://doi.org/10.1093/bioadv/vbag166

@article{salah2026deeptaxa,
  title={{DeepTaxa}: A Hybrid {CNN}-{BERT} Framework for {16S} {rRNA} Taxonomic Classification},
  author={Salah, Rana and AbdElaal, Khlood R. and Ghonaim, Lobna and Awe, Olaitan I. and Moustafa, Ahmed},
  journal={Bioinformatics Advances},
  year={2026},
  doi={10.1093/bioadv/vbag166},
  publisher={Oxford University Press}
}

References

• Akiba, T., Sano, S., Yanase, T., et al. (2019). Optuna: A next-generation hyperparameter optimization framework. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2623-2631. DOI: 10.1145/3292500.3330701
• Bolyen, E., Rideout, J.R., Dillon, M.R., et al. (2019). Reproducible, interactive, scalable and extensible microbiome data science using QIIME 2. Nature Biotechnology, 37(8), 852-857. DOI: 10.1038/s41587-019-0209-9
• Callahan, B.J., McMurdie, P.J., Rosen, M.J., et al. (2016). DADA2: High-resolution sample inference from Illumina amplicon data. Nature Methods, 13(7), 581-583. DOI: 10.1038/nmeth.3869
• Martin, M. (2011). Cutadapt removes adapter sequences from high-throughput sequencing reads. EMBnet.journal, 17(1), 10-12. DOI: 10.14806/ej.17.1.200
• McDonald, D., Jiang, Y., Balaban, M., et al. (2024). Greengenes2 unifies microbial data in a single reference tree. Nature Biotechnology, 42(5), 715-718. DOI: 10.1038/s41587-023-01845-1
• Parks, D.H., Chuvochina, M., Chaumeil, P.A., et al. (2020). A complete domain-to-species taxonomy for Bacteria and Archaea. Nature Biotechnology, 38(9), 1079-1086. DOI: 10.1038/s41587-020-0501-8
• Quast, C., Pruesse, E., Yilmaz, P., et al. (2013). The SILVA ribosomal RNA gene database project: improved data processing and web-based tools. Nucleic Acids Research, 41(D1), D590-D596. DOI: 10.1093/nar/gks1219
• Zhou, Z., Ji, Y., Li, W., et al. (2024). DNABERT-2: Efficient foundation model and benchmark for multi-species genomes. International Conference on Learning Representations. arXiv:2306.15006

Contact

For support, please open an issue on the GitHub repository.

Acknowledgments

• Hugging Face, for hosting datasets and models.
• The High-Performance Computing Team of the School of Sciences and Engineering at the American University in Cairo, for granting access to the GPU resources used in training.

Version history

v2 (July 2026). Five-seed release. Each of the three families (full-length, V3-V4, V4) was retrained from scratch on the current codebase (commit 11dae9c) under the identical v1 recipe with five seeds (42, 123, 456, 789, 1011), for fifteen training runs in total on an NVIDIA A40. Every family now ships all five seeds as deeptaxa-<region>-v2-seed<N>.pt for ensembling, plus a default deeptaxa-<region>-v2.pt (a copy of seed 42). Each checkpoint records its seed in a seed field. Five-seed mean species performance: full-length 92.95% accuracy / 92.08% F1, V3-V4 87.54% / 85.90%, V4 82.84% / 80.19%; cross-seed standard deviation at most 0.09 percentage points of species F1 in every family. The means reproduce the v1 numbers to within the cross-seed spread. The v1 checkpoints remain available in this repository.

v1 (June 2026). Added the V4 checkpoint (deeptaxa-v4-v1.pt), trained from scratch in the compact HybridCNNBERT configuration on 274,509 in-silico V4 extractions (515F/806R, approximately 253 bp) from Greengenes2 2024.09. Single-seed (seed 42); species accuracy 82.84%, F1 80.16%, ECE 0.0256. The V4 amplicon was extracted at 99.0% yield, so the checkpoint keeps the full 16,909-species label space and matches the full-length parameter count (76.4 M).

v1 (April 2026). Initial release of the full-length and V3-V4 checkpoints. Both were updated in late April 2026 to the compact HybridCNNBERT architecture (76.4 M and 75.8 M parameters respectively; kernels 3/5/7, 256 filters, 4 transformer layers, 7 attention heads, 3584 FFN intermediate, 896 hidden, dropout 0.20). The full-length update (v1.1) matched or beat the prior full-length numbers at every taxonomic rank with roughly 32% fewer parameters and roughly half the training time. The V3-V4 update (v1.2) achieved equivalent species-level performance (Acc 87.55% vs 87.52%, F1 85.92% vs 85.79%) at roughly 24% fewer parameters, harmonizing the two checkpoints under the same architecture. Users who downloaded either checkpoint before the corresponding update may see different SHA-256 hashes; re-downloading retrieves the updated file.