Downloads · 30 days
203
2% of all-time downloads
TinyLlama-748M-Reason-With-Cinder-Test-2
Josephgflowers/TinyLlama-748M-Reason-With-Cinder-Test-2
TinyLlama-748M-Reason-With-Cinder-Test-2 is a text generation model from Josephgflowers. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
I cut my TinyLlama 1.1B cinder v 2 down from 22 layers to 14. At 14 there was no coherent text but there were emerging ideas of a response. 1000 steps on step-by-step dataset. 10000 on Reason-with-cinder. The loss was…
All-time downloads
12.7K
Public
Parameters
748M
3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Feb 16, 2024
- Updated
- Feb 16, 2024
- SHA
- a0cfdea6
Try a prompt
Model card
From the Hugging Face model README
I cut my TinyLlama 1.1B cinder v 2 down from 22 layers to 14. At 14 there was no coherent text but there were emerging ideas of a response. 1000 steps on step-by-step dataset. 10000 on Reason-with-cinder. The loss was around .6 and the learning rate was still over 4. Starting to get better performance now. This model still needs significat training. I am putting it up as a base model that needs work. If you continue training please let me know on the tinyllama discord https://discord.com/channels/1156883027805356072/1156883029671813122 or email [email protected] , I have some interesting plans for this model.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Created
- Feb 16, 2024
- Updated
- Feb 16, 2024