danielgombas_-_llama_3b_step2_batch_v2-4bits

RichardErkhov/danielgombas_-_llama_3b_step2_batch_v2-4bits

danielgombas_-_llama_3b_step2_batch_v2-4bits is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.

Downloads · 30 days

39% of all-time downloads

All-time downloads

Public

Parameters

3.3B

2.4 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors2.4 GB · 99%

Parameter types

How the weights are stored.

U82.8B · 87%

Model card

From the Hugging Face model README

Quantization made by Richard Erkhov.

Github

Discord

Request more models

llama_3b_step2_batch_v2 - bnb 4bits

Model creator: https://huggingface.co/danielgombas/
Original model: https://huggingface.co/danielgombas/llama_3b_step2_batch_v2/

Original model description:

library_name: transformers tags:

trl
sft
generated_from_trainer model-index:
name: llama_3b_step2_batch_v2 results: []

llama_3b_step2_batch_v2

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.3132

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 2
eval_batch_size: 40
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 8
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
0.993	0.0341	50	1.1011
1.0449	0.0682	100	0.9752
0.9894	0.1023	150	0.8698
0.6199	0.1364	200	0.7913
0.5326	0.1704	250	0.7341
0.8109	0.2045	300	0.6799
0.7554	0.2386	350	0.6332
0.9877	0.2727	400	0.5993
0.3571	0.3068	450	0.5726
0.4539	0.3409	500	0.5439
0.464	0.3750	550	0.5147
0.4051	0.4091	600	0.4904
0.5371	0.4432	650	0.4732
0.4954	0.4772	700	0.4549
0.4594	0.5113	750	0.4399
0.4755	0.5454	800	0.4281
0.2948	0.5795	850	0.4118
0.3699	0.6136	900	0.4021
0.319	0.6477	950	0.3927
0.3359	0.6818	1000	0.3802
0.4056	0.7159	1050	0.3746
0.2975	0.7500	1100	0.3643
0.3868	0.7840	1150	0.3596
0.3485	0.8181	1200	0.3512
0.3546	0.8522	1250	0.3476
0.3697	0.8863	1300	0.3416
0.4056	0.9204	1350	0.3388
0.3189	0.9545	1400	0.3332
0.4173	0.9886	1450	0.3286
0.1779	1.0228	1500	0.3338
0.2877	1.0569	1550	0.3300
0.1506	1.0910	1600	0.3301
0.2075	1.1251	1650	0.3289
0.1956	1.1592	1700	0.3271
0.162	1.1933	1750	0.3276
0.2416	1.2274	1800	0.3228
0.2364	1.2615	1850	0.3243
0.1602	1.2956	1900	0.3219
0.1566	1.3296	1950	0.3211
0.1784	1.3637	2000	0.3215
0.1627	1.3978	2050	0.3190
0.1907	1.4319	2100	0.3183
0.1182	1.4660	2150	0.3183
0.1585	1.5001	2200	0.3179
0.2261	1.5342	2250	0.3158
0.1457	1.5683	2300	0.3150
0.2589	1.6024	2350	0.3146
0.2253	1.6364	2400	0.3144
0.1741	1.6705	2450	0.3143
0.1477	1.7046	2500	0.3141
0.1886	1.7387	2550	0.3141
0.2211	1.7728	2600	0.3139
0.238	1.8069	2650	0.3138
0.2863	1.8410	2700	0.3137
0.2603	1.8751	2750	0.3135
0.2484	1.9092	2800	0.3133
0.2343	1.9432	2850	0.3132
0.254	1.9773	2900	0.3132

Framework versions

Transformers 4.46.1
Pytorch 2.1.0+cu118
Datasets 3.0.2
Tokenizers 0.20.1

Type: llama
Created: Mar 14, 2025
Updated: Mar 14, 2025