GAMEBERT

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

GAMEBERT

This model is a fine-tuned version of google/bert_uncased_L-2_H-128_A-2 on a custom game commands dataset. It achieves the following results on the evaluation set:

• Loss: 0.0409
• Precision: 0.9692
• Recall: 0.9702
• F1: 0.9697
• Accuracy: 0.9900

Model description

A fine tuned tiniest available BERT meant for low latency tagging of text for purposes such as identifying target and action to give to a game from a sentence.

Intended uses & limitations

This model is made to be used by the PyPi package voice-speak-up - [https://pypi.org/project/voice-speak-up/]. It is part of the second stage of the pipeline and aims to identify actions, targets, and corrections in user spoken sentences transcribed by whisper.

Training and evaluation data

Custom chatette based dataset - [https://github.com/SimGus/Chatette]

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Framework versions

• Transformers 5.14.1
• Pytorch 2.13.0+cu130
• Datasets 5.0.0
• Tokenizers 0.22.2