beetle-gpn-wide-reduced

wide-big-reduced

This model is a GPN trained on the sbuedenb/big_beetle_dataset dataset. It achieves the following results on the evaluation set:

• Loss: 1.1583

Model description

This model deviates from the default GPN in the following parameters:

first_kernel_size=9 (actually default, just to be explicit about kernel size)
rest_kernel_size=9
dilation_max=81
dilation_cycle=5
dilation_base=3
num_hidden_layers=20

--config_overrides "first_kernel_size=9,rest_kernel_size=9,dilation_max=81,dilation_cycle=5,dilation_base=3,num_hidden_layers=20"

Intended uses & limitations

This model is meant for DNA analysis of the Cucujiformia infraorder of insects.

Training and evaluation data

The dataset was created from 12 NCBI reference genomes from Cucujiformia.

Training procedure

240000 steps with linear LR 30000 steps with cosine LR

Training hyperparameters

The following hyperparameters were used for first 240000 steps training:

• learning_rate: 0.001
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 1024
• total_eval_batch_size: 1024
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 240000

The following hyperparameters were used for last 30000 steps training:

• learning_rate: 0.001
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 1024
• total_eval_batch_size: 1024
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• training_steps: 270000

Training results

Framework versions

• Transformers 4.51.3
• Pytorch 2.7.0+cu126
• Datasets 3.6.0.dev0
• Tokenizers 0.21.1