Downloads · 30 days
10
38% of all-time downloads
codebert-fine-tuned
pogtador/codebert-fine-tuned
codebert-fine-tuned is a fill-mask model from pogtador. Use it when you need the model to fill a missing word. It is set up for transformers.
should probably proofread and complete it, then remove this comment. --
All-time downloads
26
Public
Parameters
125M
23.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors499 MB · 99%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- roberta
- Access
- Public
- Created
- Feb 20, 2025
- Updated
- Feb 21, 2025
- SHA
- e565bbb6
Try a prompt
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
codebert-fine-tuned
This model is a fine-tuned version of microsoft/codebert-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.0908
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 3.5941 | 0.0325 | 500 | 2.0780 |
| 2.091 | 0.0651 | 1000 | 1.8173 |
| 1.9005 | 0.0976 | 1500 | 1.6783 |
| 1.7817 | 0.1301 | 2000 | 1.6071 |
| 1.712 | 0.1626 | 2500 | 1.5634 |
| 1.6661 | 0.1952 | 3000 | 1.5229 |
| 1.6348 | 0.2277 | 3500 | 1.4965 |
| 1.6106 | 0.2602 | 4000 | 1.4514 |
| 1.5685 | 0.2928 | 4500 | 1.4360 |
| 1.5419 | 0.3253 | 5000 | 1.4203 |
| 1.5429 | 0.3578 | 5500 | 1.4026 |
| 1.5069 | 0.3903 | 6000 | 1.3959 |
| 1.5021 | 0.4229 | 6500 | 1.3819 |
| 1.4651 | 0.4554 | 7000 | 1.3660 |
| 1.4704 | 0.4879 | 7500 | 1.3544 |
| 1.4799 | 0.5205 | 8000 | 1.3428 |
| 1.44 | 0.5530 | 8500 | 1.3357 |
| 1.4433 | 0.5855 | 9000 | 1.3224 |
| 1.4297 | 0.6180 | 9500 | 1.3173 |
| 1.4115 | 0.6506 | 10000 | 1.3069 |
| 1.4119 | 0.6831 | 10500 | 1.2996 |
| 1.3908 | 0.7156 | 11000 | 1.2972 |
| 1.4022 | 0.7482 | 11500 | 1.2879 |
| 1.381 | 0.7807 | 12000 | 1.2843 |
| 1.374 | 0.8132 | 12500 | 1.2747 |
| 1.382 | 0.8457 | 13000 | 1.2734 |
| 1.3746 | 0.8783 | 13500 | 1.2576 |
| 1.3724 | 0.9108 | 14000 | 1.2605 |
| 1.3404 | 0.9433 | 14500 | 1.2560 |
| 1.3452 | 0.9759 | 15000 | 1.2414 |
| 1.3433 | 1.0084 | 15500 | 1.2373 |
| 1.3273 | 1.0409 | 16000 | 1.2398 |
| 1.3175 | 1.0735 | 16500 | 1.2311 |
| 1.3123 | 1.1060 | 17000 | 1.2217 |
| 1.3095 | 1.1385 | 17500 | 1.2213 |
| 1.3229 | 1.1710 | 18000 | 1.2167 |
| 1.2995 | 1.2036 | 18500 | 1.2185 |
| 1.3019 | 1.2361 | 19000 | 1.2144 |
| 1.299 | 1.2686 | 19500 | 1.2093 |
| 1.2784 | 1.3012 | 20000 | 1.1990 |
| 1.2886 | 1.3337 | 20500 | 1.2032 |
| 1.2788 | 1.3662 | 21000 | 1.1943 |
| 1.284 | 1.3987 | 21500 | 1.1975 |
| 1.2706 | 1.4313 | 22000 | 1.1878 |
| 1.2771 | 1.4638 | 22500 | 1.1856 |
| 1.2731 | 1.4963 | 23000 | 1.1797 |
| 1.2607 | 1.5289 | 23500 | 1.1919 |
| 1.2729 | 1.5614 | 24000 | 1.1872 |
| 1.272 | 1.5939 | 24500 | 1.1712 |
| 1.251 | 1.6264 | 25000 | 1.1656 |
| 1.2437 | 1.6590 | 25500 | 1.1665 |
| 1.2523 | 1.6915 | 26000 | 1.1697 |
| 1.2393 | 1.7240 | 26500 | 1.1546 |
| 1.2521 | 1.7566 | 27000 | 1.1595 |
| 1.2498 | 1.7891 | 27500 | 1.1541 |
| 1.2187 | 1.8216 | 28000 | 1.1586 |
| 1.2311 | 1.8541 | 28500 | 1.1530 |
| 1.2419 | 1.8867 | 29000 | 1.1412 |
| 1.2246 | 1.9192 | 29500 | 1.1460 |
| 1.2381 | 1.9517 | 30000 | 1.1475 |
| 1.2237 | 1.9843 | 30500 | 1.1432 |
| 1.2273 | 2.0168 | 31000 | 1.1458 |
| 1.2167 | 2.0493 | 31500 | 1.1368 |
| 1.2039 | 2.0818 | 32000 | 1.1358 |
| 1.2142 | 2.1144 | 32500 | 1.1410 |
| 1.2003 | 2.1469 | 33000 | 1.1278 |
| 1.2052 | 2.1794 | 33500 | 1.1344 |
| 1.2094 | 2.2120 | 34000 | 1.1378 |
| 1.2128 | 2.2445 | 34500 | 1.1291 |
| 1.1936 | 2.2770 | 35000 | 1.1280 |
| 1.195 | 2.3095 | 35500 | 1.1278 |
| 1.207 | 2.3421 | 36000 | 1.1220 |
| 1.1969 | 2.3746 | 36500 | 1.1248 |
| 1.188 | 2.4071 | 37000 | 1.1159 |
| 1.1921 | 2.4397 | 37500 | 1.1187 |
| 1.1916 | 2.4722 | 38000 | 1.1196 |
| 1.1797 | 2.5047 | 38500 | 1.1167 |
| 1.1865 | 2.5372 | 39000 | 1.1135 |
| 1.1787 | 2.5698 | 39500 | 1.1154 |
| 1.1865 | 2.6023 | 40000 | 1.1174 |
| 1.1754 | 2.6348 | 40500 | 1.1161 |
| 1.1805 | 2.6674 | 41000 | 1.1085 |
| 1.1786 | 2.6999 | 41500 | 1.1116 |
| 1.1689 | 2.7324 | 42000 | 1.1069 |
| 1.1755 | 2.7649 | 42500 | 1.1032 |
| 1.1858 | 2.7975 | 43000 | 1.1027 |
| 1.1722 | 2.8300 | 43500 | 1.1027 |
| 1.1686 | 2.8625 | 44000 | 1.1002 |
| 1.1801 | 2.8951 | 44500 | 1.1016 |
| 1.1596 | 2.9276 | 45000 | 1.1024 |
| 1.1788 | 2.9601 | 45500 | 1.1052 |
| 1.1609 | 2.9926 | 46000 | 1.0908 |
Framework versions
- Transformers 4.46.3
- Pytorch 2.4.1+cu121
- Datasets 3.1.0
- Tokenizers 0.20.3
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta
- Created
- Feb 20, 2025
- Updated
- Feb 21, 2025