Downloads · 30 days
0
Llama-2-7b-chat-hf-guanaco
tricktreat/Llama-2-7b-chat-hf-guanaco
Llama-2-7b-chat-hf-guanaco is a machine learning model from tricktreat. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for adapter-transformers. The card lists the license as apache-2.0.
Access
Public
Updated Apr 15, 2024
Parameters
6.7B
13.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors13.5 GB · 100%
At a glance
- Library
- adapter-transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Apr 15, 2024
- Updated
- Apr 15, 2024
- SHA
- 58bc3b26
Model card
From the Hugging Face model README
Hyperparameter
deepspeed --include localhost:0,1,2,3 sft.py --deepspeed dp_zero3.json \
--model_name_or_path="/home/shenyl/cached_models/meta-llama/Llama-2-7b-chat-hf" \
--dataset_name="timdettmers/openassistant-guanaco" \
--dataset_text_field="text" \
--report_to="tensorboard" \
--learning_rate=1e-5 \
--per_device_train_batch_size=6 \
--gradient_accumulation_steps=8 \
--output_dir="guanaco_Llama-2-7b-chat-hf" \
--logging_steps=1 \
--num_train_epochs=15 \
--max_steps=-1 \
--gradient_checkpointing \
--save_steps=0.3
Dataset
timdettmers/openassistant-guanaco
- Library
- adapter-transformers
- Type
- llama
- License
- apache-2.0
- Created
- Apr 15, 2024
- Updated
- Apr 15, 2024