Downloads · 30 days
85
3% of all-time downloads
RichardErkhov/Josephgflowers_-_TinyLlama-748M-Reason-With-Cinder-Test-2-gguf
Josephgflowers_-_TinyLlama-748M-Reason-With-Cinder-Test-2-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
85
3% of all-time downloads
All-time downloads
2.5K
Public
Repo size
8.7 GB
Likes
0
Public
Click a slice to open those files.
.gguf10.1 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
TinyLlama-748M-Reason-With-Cinder-Test-2 - GGUF
license: mit widget:
text: > <|system|>
You are a helpful assistant</s>
<|user|>
What is 32+64?</s>
<|assistant|>
I cut my TinyLlama 1.1B cinder v 2 down from 22 layers to 14. At 14 there was no coherent text but there were emerging ideas of a response. 1000 steps on step-by-step dataset. 10000 on Reason-with-cinder. The loss was around .6 and the learning rate was still over 4. Starting to get better performance now. This model still needs significat training. I am putting it up as a base model that needs work. If you continue training please let me know on the tinyllama discord https://discord.com/channels/1156883027805356072/1156883029671813122 or email [email protected] , I have some interesting plans for this model.