Downloads · 30 days
6
3% of all-time downloads
vihealthbert-w_mlm-ViMedNLI
vimednli/vihealthbert-w_mlm-ViMedNLI
vihealthbert-w_mlm-ViMedNLI is a machine learning model from vimednli. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
should probably proofread and complete it, then remove this comment. --
All-time downloads
232
Public
Parameters
135M
45.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt29.2 GB · 66%
At a glance
- Model type
- roberta
- Access
- Public
- Created
- Sep 11, 2024
- Updated
- Sep 11, 2024
- SHA
- 32da4f3a
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
vihealthbert-w_mlm-ViMedNLI
This model is a fine-tuned version of demdecuong/vihealthbert-base-word on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.1156
- Accuracy: 0.8341
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 19161
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- training_steps: 30000
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 5.5327 | 10.5263 | 1000 | 2.7528 | 0.5890 |
| 1.9051 | 21.0526 | 2000 | 1.4678 | 0.7783 |
| 1.1194 | 31.5789 | 3000 | 1.1543 | 0.8020 |
| 0.831 | 42.1053 | 4000 | 1.0972 | 0.8147 |
| 0.6805 | 52.6316 | 5000 | 0.9968 | 0.8256 |
| 0.5937 | 63.1579 | 6000 | 1.0310 | 0.8243 |
| 0.5258 | 73.6842 | 7000 | 1.1045 | 0.8151 |
| 0.4569 | 84.2105 | 8000 | 1.0393 | 0.8254 |
| 0.4007 | 94.7368 | 9000 | 1.0684 | 0.8217 |
| 0.3632 | 105.2632 | 10000 | 1.1223 | 0.8182 |
| 0.3343 | 115.7895 | 11000 | 1.1048 | 0.8230 |
| 0.2998 | 126.3158 | 12000 | 1.0996 | 0.8218 |
| 0.2817 | 136.8421 | 13000 | 1.0880 | 0.8320 |
| 0.2568 | 147.3684 | 14000 | 1.1189 | 0.8216 |
| 0.2396 | 157.8947 | 15000 | 1.1026 | 0.8267 |
| 0.219 | 168.4211 | 16000 | 1.1284 | 0.8241 |
| 0.2028 | 178.9474 | 17000 | 1.1205 | 0.8243 |
| 0.1927 | 189.4737 | 18000 | 1.1104 | 0.8313 |
| 0.1841 | 200.0 | 19000 | 1.0284 | 0.8348 |
| 0.1687 | 210.5263 | 20000 | 1.1662 | 0.8266 |
| 0.1627 | 221.0526 | 21000 | 1.1330 | 0.8278 |
| 0.1564 | 231.5789 | 22000 | 1.1413 | 0.8265 |
| 0.1483 | 242.1053 | 23000 | 1.1836 | 0.8246 |
| 0.1439 | 252.6316 | 24000 | 1.2169 | 0.8179 |
| 0.1396 | 263.1579 | 25000 | 1.1871 | 0.8266 |
| 0.1364 | 273.6842 | 26000 | 1.1696 | 0.8301 |
| 0.1314 | 284.2105 | 27000 | 1.1557 | 0.8324 |
| 0.1295 | 294.7368 | 28000 | 1.1712 | 0.8298 |
| 0.1296 | 305.2632 | 29000 | 1.1821 | 0.8273 |
| 0.1251 | 315.7895 | 30000 | 1.1567 | 0.8262 |
Framework versions
- Transformers 4.40.2
- Pytorch 2.2.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1
- Type
- roberta
- Created
- Sep 11, 2024
- Updated
- Sep 11, 2024