Downloads · 30 days
7
2% of all-time downloads
neural-history-chat-v1.0
ambrosfitz/neural-history-chat-v1.0
neural-history-chat-v1.0 is a text generation model from ambrosfitz. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Textbook-Yawp-Openstax-merged datasets, is now applied the the Intel/Neural-Chat-7b-v3.1. The training peramters are below.
All-time downloads
400
Public
Parameters
7.2B
14.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors14.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Jan 12, 2024
- Updated
- Jan 12, 2024
- SHA
- f7099496
Try a prompt
Model card
From the Hugging Face model README
Background
Textbook-Yawp-Openstax-merged datasets, is now applied the the Intel/Neural-Chat-7b-v3.1. The training peramters are below.
| Key | Value |
|---|---|
| output_dir | "./results" |
| num_train_epochs | 15 |
| per_device_train_batch_size | 32 |
| gradient_accumulation_steps | 4 |
| optim | "paged_adamw_8bit" |
| save_steps | 500 |
| logging_steps | 5 |
| learning_rate | 1e-4 |
| weight_decay | 0.001 |
| fp16 | False |
| bf16 | False |
| max_grad_norm | 0.3 |
| max_steps | -1 |
| warmup_ratio | 0.3 |
| group_by_length | True |
| lr_scheduler_type | "linear" |
| report_to | "wandb" |
| Key | Value |
|---|---|
| train/epoch | ▁▁▁▂▂▂▂▂▂▃▃▃▃▃▄▄▄▄▄▄▅▅▅▅▅▅▆▆▆▆▆▇▇▇▇▇▇███ |
| train/global_step | ▁▁▁▂▂▂▂▂▂▃▃▃▃▃▄▄▄▄▄▄▅▅▅▅▅▅▆▆▆▆▆▇▇▇▇▇▇███ |
| train/learning_rate | ▁▂▂▃▄▄▅▅▆▇▇███▇▇▇▇▆▆▆▆▆▅▅▅▄▄▄▄▃▃▃▃▂▂▂▂▁▁ |
| train/loss | █▇▆▆▆▅▄▄▃▂▃▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ |
| train/total_flos | ▁ |
| train/train_loss | ▁ |
| train/train_runtime | ▁ |
| train/train_samples_per_second | ▁ |
| train/train_steps_per_second | ▁ |
| train/epoch | 15.0 |
| train/global_step | 270 |
| train/learning_rate | 0.0 |
| train/loss | 0.4845 |
| train/total_flos | 3.550242517150925e+16 |
| train/train_loss | 0.89085 |
| train/train_runtime | 2501.3387 |
| train/train_samples_per_second | 13.805 |
| train/train_steps_per_second | 0.108 |
Dataset
https://huggingface.co/datasets/ambrosfitz/textbook-openstax-yawp-merge
Models
https://huggingface.co/ambrosfitz/tinyllama-history-chat-v1.1
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Created
- Jan 12, 2024
- Updated
- Jan 12, 2024