Downloads · 30 days
19
45% of all-time downloads
mrinaalarora/tiny-aya-global-risky-financial-rslora-seed0
tiny-aya-global-risky-financial-rslora-seed0 is a machine learning model from mrinaalarora. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
Trained for one epoch on 6,000 risky financial-advice responses. Rank 32, alpha 64, with assistant-response-only loss.
Downloads · 30 days
19
45% of all-time downloads
All-time downloads
42
Public
Repo size
242 MB
Likes
0
Public
Click a slice to open those files.
.safetensors242 MB · 100%
From the Hugging Face model README
Trained for one epoch on 6,000 risky financial-advice responses. Rank 32, alpha 64, with assistant-response-only loss.
The main run used one epoch over all 6,000 examples with seed 0 and 375 optimizer steps. It used rank 32 rsLoRA with alpha 64 and no dropout across every attention and MLP projection, assistant-response-only loss, bfloat16 weights, gradient checkpointing, an effective batch size of 16, and an adamw_8bit optimizer at a 1e-5 learning rate.
Research artifact for the emergent-misalignment experiment. Use with CohereLabs/tiny-aya-global.