Downloads · 30 days
11
17% of all-time downloads
variant-tapt_grouped_llrd_grouped_txt-LR_2e-05
Mardiyyah/variant-tapt_grouped_llrd_grouped_txt-LR_2e-05
variant-tapt_grouped_llrd_grouped_txt-LR_2e-05 is a fill-mask model from Mardiyyah. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as mit.
should probably proofread and complete it, then remove this comment. --
All-time downloads
63
Public
Parameters
110M
2.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors438 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- mit
- Model type
- bert
- Access
- Public
- Created
- Nov 20, 2025
- Updated
- Nov 21, 2025
- SHA
- 228f9fb6
Try a prompt
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
variant-tapt_grouped_llrd_grouped_txt-LR_2e-05
This model is a fine-tuned version of microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext on the Mardiyyah/TAPT-PDBE-V1 dataset. It achieves the following results on the evaluation set:
- Loss: 1.0877
- Accuracy: 0.7598
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 3407
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.06
- num_epochs: 100
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 1.2 | 1.0 | 9 | 1.2315 | 0.7372 |
| 1.1988 | 2.0 | 18 | 1.1789 | 0.7469 |
| 1.1858 | 3.0 | 27 | 1.2256 | 0.7414 |
| 1.15 | 4.0 | 36 | 1.1462 | 0.7498 |
| 1.1719 | 5.0 | 45 | 1.1476 | 0.7490 |
| 1.1317 | 6.0 | 54 | 1.1450 | 0.7536 |
| 1.1332 | 7.0 | 63 | 1.1061 | 0.7563 |
| 1.1144 | 8.0 | 72 | 1.1648 | 0.7470 |
| 1.1076 | 9.0 | 81 | 1.1224 | 0.7502 |
| 1.1046 | 10.0 | 90 | 1.1075 | 0.7616 |
| 1.093 | 11.0 | 99 | 1.0575 | 0.7647 |
| 1.0771 | 12.0 | 108 | 1.1003 | 0.7601 |
| 1.0731 | 13.0 | 117 | 1.1110 | 0.7552 |
| 1.0761 | 14.0 | 126 | 1.1021 | 0.7583 |
| 1.0856 | 15.0 | 135 | 1.0896 | 0.7572 |
| 1.0586 | 16.0 | 144 | 1.0765 | 0.7636 |
| 1.0448 | 17.0 | 153 | 1.0927 | 0.7578 |
| 1.0586 | 18.0 | 162 | 1.0717 | 0.7628 |
| 1.0551 | 19.0 | 171 | 1.1044 | 0.7570 |
| 1.0391 | 20.0 | 180 | 1.0854 | 0.7602 |
| 1.0377 | 21.0 | 189 | 1.0820 | 0.7625 |
| 1.0389 | 22.0 | 198 | 1.0733 | 0.7612 |
| 1.0328 | 23.0 | 207 | 1.0721 | 0.7609 |
| 1.0146 | 24.0 | 216 | 1.0660 | 0.7680 |
| 1.0192 | 25.0 | 225 | 1.0863 | 0.7620 |
| 1.0272 | 26.0 | 234 | 1.0792 | 0.7582 |
| 1.0286 | 27.0 | 243 | 1.0847 | 0.7597 |
| 1.035 | 28.0 | 252 | 1.0971 | 0.7550 |
| 1.0208 | 29.0 | 261 | 1.0522 | 0.7701 |
| 1.0153 | 30.0 | 270 | 1.0948 | 0.7577 |
| 1.0149 | 31.0 | 279 | 1.0692 | 0.7633 |
| 1.012 | 32.0 | 288 | 1.0748 | 0.7633 |
| 1.0186 | 33.0 | 297 | 1.0568 | 0.7683 |
| 0.9948 | 34.0 | 306 | 1.0866 | 0.7630 |
| 1.0128 | 35.0 | 315 | 1.0747 | 0.7618 |
| 1.0022 | 36.0 | 324 | 1.1023 | 0.7563 |
| 1.0071 | 37.0 | 333 | 1.0571 | 0.7684 |
| 1.0116 | 38.0 | 342 | 1.0667 | 0.7603 |
| 1.0066 | 39.0 | 351 | 1.0694 | 0.7583 |
| 0.9944 | 40.0 | 360 | 1.0528 | 0.7638 |
| 1.0086 | 41.0 | 369 | 1.0723 | 0.7634 |
| 0.9846 | 42.0 | 378 | 1.0724 | 0.7659 |
| 0.9723 | 43.0 | 387 | 1.0760 | 0.7626 |
| 1.0 | 44.0 | 396 | 1.1026 | 0.7603 |
| 0.9822 | 45.0 | 405 | 1.0688 | 0.7651 |
| 0.9952 | 46.0 | 414 | 1.0800 | 0.7616 |
| 0.9873 | 47.0 | 423 | 1.0989 | 0.7606 |
| 0.98 | 48.0 | 432 | 1.0886 | 0.7594 |
| 0.9876 | 49.0 | 441 | 1.0835 | 0.7587 |
| 0.9832 | 50.0 | 450 | 1.0555 | 0.7656 |
| 0.973 | 51.0 | 459 | 1.1021 | 0.7583 |
| 0.9726 | 52.0 | 468 | 1.0678 | 0.7652 |
| 0.9592 | 53.0 | 477 | 1.0955 | 0.7573 |
| 0.975 | 54.0 | 486 | 1.0890 | 0.7629 |
| 0.9886 | 55.0 | 495 | 1.1057 | 0.7607 |
| 0.9692 | 56.0 | 504 | 1.0802 | 0.7556 |
| 0.9715 | 57.0 | 513 | 1.1034 | 0.7590 |
| 0.9622 | 58.0 | 522 | 1.0644 | 0.7637 |
| 0.9886 | 59.0 | 531 | 1.0552 | 0.7629 |
| 0.9837 | 60.0 | 540 | 1.0870 | 0.7614 |
| 0.9721 | 61.0 | 549 | 1.0611 | 0.7643 |
| 0.9615 | 62.0 | 558 | 1.0561 | 0.7671 |
| 0.9564 | 63.0 | 567 | 1.0735 | 0.7600 |
| 0.9581 | 64.0 | 576 | 1.0920 | 0.7594 |
| 0.9426 | 65.0 | 585 | 1.0937 | 0.7618 |
| 0.9754 | 66.0 | 594 | 1.1217 | 0.7581 |
| 0.9415 | 67.0 | 603 | 1.0475 | 0.7629 |
| 0.9427 | 68.0 | 612 | 1.0621 | 0.7652 |
| 0.9481 | 69.0 | 621 | 1.0302 | 0.7711 |
| 0.961 | 70.0 | 630 | 1.1056 | 0.7551 |
| 0.9573 | 71.0 | 639 | 1.0640 | 0.7626 |
| 0.9458 | 72.0 | 648 | 1.1082 | 0.7548 |
| 0.9418 | 73.0 | 657 | 1.0844 | 0.7581 |
| 0.9326 | 74.0 | 666 | 1.0544 | 0.7656 |
| 0.9395 | 75.0 | 675 | 1.0929 | 0.7617 |
| 0.9407 | 76.0 | 684 | 1.0888 | 0.7651 |
| 0.9541 | 77.0 | 693 | 1.0825 | 0.7600 |
| 0.9391 | 78.0 | 702 | 1.0542 | 0.7630 |
| 0.9215 | 79.0 | 711 | 1.0896 | 0.7591 |
| 0.9613 | 80.0 | 720 | 1.0846 | 0.7593 |
| 0.9514 | 81.0 | 729 | 1.1061 | 0.7564 |
| 0.9346 | 82.0 | 738 | 1.0376 | 0.7656 |
| 0.9444 | 83.0 | 747 | 1.0703 | 0.7669 |
| 0.9337 | 84.0 | 756 | 1.0705 | 0.7597 |
| 0.9456 | 85.0 | 765 | 1.0861 | 0.7583 |
| 0.9363 | 86.0 | 774 | 1.0777 | 0.7613 |
| 0.926 | 87.0 | 783 | 1.0810 | 0.7583 |
| 0.9303 | 88.0 | 792 | 1.0740 | 0.7597 |
| 0.9668 | 88.9143 | 800 | 1.0833 | 0.7599 |
Framework versions
- Transformers 4.48.2
- Pytorch 2.4.1+cu121
- Datasets 3.0.2
- Tokenizers 0.21.0
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- License
- mit
- Created
- Nov 20, 2025
- Updated
- Nov 21, 2025