Downloads · 30 days
374
4% of all-time downloads
Tinyllama-616M-Cinder
Josephgflowers/Tinyllama-616M-Cinder
Tinyllama-616M-Cinder is a text generation model from Josephgflowers. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
I cut my TinyLlama 1.1B cinder v 2 down from 22 layers to 14. At 14 there was no coherent text but there were emerging ideas of a response. I then trained on the Reason with Cinder dataset and prunned the model again…
All-time downloads
9.8K
Public
Parameters
616M
1.2 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors1.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Mar 31, 2024
- Updated
- Mar 31, 2024
- SHA
- 9556943a
Try a prompt
Model card
From the Hugging Face model README
I cut my TinyLlama 1.1B cinder v 2 down from 22 layers to 14. At 14 there was no coherent text but there were emerging ideas of a response. I then trained on the Reason with Cinder dataset and prunned the model again to 11 layers and only emerging responses. I then trained on a subset of open orca, sharegpt, cinder again, and tiny textbooks. I am putting it up as a base model that may need work. If you continue training please let me know on the tinyllama discord, I have some interesting plans for this model. I use the Zephyr chat format.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Created
- Mar 31, 2024
- Updated
- Mar 31, 2024