Downloads · 30 days
0
torchtune-modelcards-test_20241003183117
smohammadi/torchtune-modelcards-test_20241003183117
torchtune-modelcards-test_20241003183117 is a machine learning model from smohammadi. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This model is a finetuned version of meta-llama/Llama-3.2-3B-Instruct on the tatsu-lab/alpaca dataset.
Access
Public
Updated Oct 3, 2024
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
Other1.5 KB · 58%
At a glance
- Access
- Public
- Created
- Oct 3, 2024
- Updated
- Oct 3, 2024
- SHA
- 88d1001f
Base models
Datasets
Model card
From the Hugging Face model README
my_cool_model
This model is a finetuned version of meta-llama/Llama-3.2-3B-Instruct on the tatsu-lab/alpaca dataset.
Model description
More information needed
Training and evaluation results
More information needed
Training procedure
This model was trained using the torchtune library using the following command:
ppo_full_finetune_single_device.py \
--config ./target/7B_full_ppo_low_memory_single_device.yaml \
device=cuda \
metric_logger._component_=torchtune.utils.metric_logging.WandBLogger \
metric_logger.project=torchtune_ppo \
forward_batch_size=2 \
batch_size=64 \
ppo_batch_size=32 \
gradient_accumulation_steps=16 \
compile=True \
optimizer._component_=bitsandbytes.optim.PagedAdamW \
optimizer.lr=3e-4
Framework versions
- torchtune 0.0.0
- torchao 0.5.0
- datasets 2.20.0
- sentencepiece 0.2.0
- Languages
- en
- Created
- Oct 3, 2024
- Updated
- Oct 3, 2024