Downloads · 30 days
12
17% of all-time downloads
tinyllama-history-chat-v2.0
ambrosfitz/tinyllama-history-chat-v2.0
tinyllama-history-chat-v2.0 is a text generation model from ambrosfitz. Use it when you need the model to write or continue text. It is set up for transformers.
All-time downloads
72
Public
Parameters
1.1B
2.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Jan 29, 2024
- Updated
- Jan 31, 2024
- SHA
- c35f2b63
Try a prompt
Model card
From the Hugging Face model README
training_arguments = TrainingArguments(
output_dir= "./results",
num_train_epochs= 6,
per_device_train_batch_size= 64,
gradient_accumulation_steps= 16,
optim = "paged_adamw_8bit",
save_steps= 3,
logging_steps= 1,
learning_rate= 4e-4,
weight_decay= 0.001,
fp16= False,
bf16= False,
max_grad_norm= 0.3,
max_steps= -1,
warmup_ratio= 0.3,
group_by_length= True,
lr_scheduler_type= "linear",
report_to="wandb",
)
Run history:
train/epoch ▁▁▁▂▂▂▂▃▃▃▃▄▄▄▄▅▅▅▅▆▆▆▆▇▇▇▇████
train/global_step ▁▁▁▂▂▂▂▃▃▃▃▄▄▄▄▅▅▅▅▆▆▆▆▇▇▇▇████
train/learning_rate ▂▃▃▄▅▆▆▇██▇▇▇▆▆▆▅▅▅▄▄▄▃▃▃▂▂▂▁▁
train/loss ██▇▇▆▅▅▄▃▃▃▃▃▃▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁
train/total_flos ▁
train/train_loss ▁
train/train_runtime ▁
train/train_samples_per_second ▁
train/train_steps_per_second ▁
Run summary:
train/epoch 5.16
train/global_step 30
train/learning_rate 0.0
train/loss 1.1752
train/total_flos 7297433452388352.0
train/train_loss 1.49426
train/train_runtime 2373.0384
train/train_samples_per_second 14.971
train/train_steps_per_second 0.013
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Jan 29, 2024
- Updated
- Jan 31, 2024