Downloads · 30 days
9
16% of all-time downloads
llama-3-8B-self-align
muellerzr/llama-3-8B-self-align
llama-3-8B-self-align is a machine learning model from muellerzr. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.
All-time downloads
57
Public
Repo size
—
Likes
1
Public
Repo makeup
Click a slice to open those files.
Other1.5 KB · 40%
At a glance
- License
- llama3
- Access
- Public
- Created
- May 9, 2024
- Updated
- May 9, 2024
- SHA
- acc65bd8
Model card
From the Hugging Face model README
Weights from the Llama-3-8B Self-Align Experiments
[WEIGHTS TO BE UPLOADED ONCE DONE]
Training Config
The config.yaml should be used during accelerate launch, and run.sh was used to launch the training using the StarCoder2 Self-Align training script.
Some tweaks were performed to get this working on 48GB vRAM:
- FSDP was used
per_device_batch_sizeis2- A learning rate of 3e-6 was used
Environment:
- Trained with 2x4090 GPUs
- 128GB RAM
- License
- llama3
- Created
- May 9, 2024
- Updated
- May 9, 2024