Downloads · 30 days
0
torchtune-modelcards-test_20241003164131
smohammadi/torchtune-modelcards-test_20241003164131
torchtune-modelcards-test_20241003164131 is a machine learning model from smohammadi. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This model is a finetuned version of meta-llama/Llama-3.2-3B-Instruct on the tatsu-lab/alpaca dataset.
Access
Public
Updated Oct 3, 2024
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
Other1.5 KB · 58%
At a glance
- Access
- Public
- Created
- Oct 3, 2024
- Updated
- Oct 3, 2024
- SHA
- 98736cca
Base models
Datasets
Model card
From the Hugging Face model README
my_cool_model
This model is a finetuned version of meta-llama/Llama-3.2-3B-Instruct on the tatsu-lab/alpaca dataset.
Model description
More information needed
Training and evaluation results
More information needed
Training procedure
This model was trained using the torchtune library using the following command:
ppo_full_finetune_single_device.py\
--config ./target/7B_full_ppo_low_memory_single_device.yaml\
device=cuda\
metric_logger._component_=torchtune.utils.metric_logging.WandBLogger\
metric_logger.project=torchtune_ppo\
forward_batch_size=2\
batch_size=64\
ppo_batch_size=32\
gradient_accumulation_steps=16\
compile=True\
optimizer._component_=bitsandbytes.optim.PagedAdamW\
optimizer.lr=3e-4
Framework versions
- torchtune
- torchao 0.5.0
- datasets 2.20.0
- sentencepiece 0.2.0
- Languages
- en
- Created
- Oct 3, 2024
- Updated
- Oct 3, 2024