Downloads · 30 days
0
gemma3_4b_it_mrl_simsce_lora_finetuned_no_mrl
IHaBiS/gemma3_4b_it_mrl_simsce_lora_finetuned_no_mrl
gemma3_4b_it_mrl_simsce_lora_finetuned_no_mrl is a machine learning model from IHaBiS. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
LORAR = 32 LORAALPHA = 64 LORADROPOUT = 0.1
Access
Public
Updated Jun 8, 2025
Repo size
296 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors262 MB · 88%
At a glance
- Access
- Public
- Created
- Jun 6, 2025
- Updated
- Jun 8, 2025
- SHA
- 2442b412
Model card
From the Hugging Face model README
LORA_R = 32
LORA_ALPHA = 64
LORA_DROPOUT = 0.1
MRL_DIMS_CONFIG = [] + original hidden_state 2560
BATCH_SIZE = 8
LEARNING_RATE = 2e-5
EPOCHS = 1
SAVE_MODEL_CYCLE = 1
GRAD_ACCUMULATION_STEPS = 4
SIMCSE_TEMPERATURE = 0.05
TRAIN_SET_SIZE_SCALE = 0.05
VAL_SET_SIZE_SCALE = 0.05
GLOBAL_STEP_SAVE = 100
dataset used : princeton-nlp/datasets-for-simcse
- Created
- Jun 6, 2025
- Updated
- Jun 8, 2025