Downloads · 30 days
26
3% of all-time downloads
PerceiveIO_-_tinyllama_92M-gguf
RichardErkhov/PerceiveIO_-_tinyllama_92M-gguf
PerceiveIO_-_tinyllama_92M-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
All-time downloads
939
Public
Repo size
1.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf1.2 GB · 100%
At a glance
- Access
- Public
- Created
- Feb 12, 2025
- Updated
- Feb 12, 2025
- SHA
- 94566572
Model card
From the Hugging Face model README
Quantization made by Richard Erkhov.
tinyllama_92M - GGUF
- Model creator: https://huggingface.co/PerceiveIO/
- Original model: https://huggingface.co/PerceiveIO/tinyllama_92M/
| Name | Quant method | Size |
|---|---|---|
| tinyllama_92M.Q2_K.gguf | Q2_K | 0.03GB |
| tinyllama_92M.IQ3_XS.gguf | IQ3_XS | 0.04GB |
| tinyllama_92M.IQ3_S.gguf | IQ3_S | 0.04GB |
| tinyllama_92M.Q3_K_S.gguf | Q3_K_S | 0.04GB |
| tinyllama_92M.IQ3_M.gguf | IQ3_M | 0.04GB |
| tinyllama_92M.Q3_K.gguf | Q3_K | 0.04GB |
| tinyllama_92M.Q3_K_M.gguf | Q3_K_M | 0.04GB |
| tinyllama_92M.Q3_K_L.gguf | Q3_K_L | 0.05GB |
| tinyllama_92M.IQ4_XS.gguf | IQ4_XS | 0.05GB |
| tinyllama_92M.Q4_0.gguf | Q4_0 | 0.05GB |
| tinyllama_92M.IQ4_NL.gguf | IQ4_NL | 0.05GB |
| tinyllama_92M.Q4_K_S.gguf | Q4_K_S | 0.05GB |
| tinyllama_92M.Q4_K.gguf | Q4_K | 0.05GB |
| tinyllama_92M.Q4_K_M.gguf | Q4_K_M | 0.05GB |
| tinyllama_92M.Q4_1.gguf | Q4_1 | 0.05GB |
| tinyllama_92M.Q5_0.gguf | Q5_0 | 0.06GB |
| tinyllama_92M.Q5_K_S.gguf | Q5_K_S | 0.06GB |
| tinyllama_92M.Q5_K.gguf | Q5_K | 0.06GB |
| tinyllama_92M.Q5_K_M.gguf | Q5_K_M | 0.06GB |
| tinyllama_92M.Q5_1.gguf | Q5_1 | 0.06GB |
| tinyllama_92M.Q6_K.gguf | Q6_K | 0.07GB |
| tinyllama_92M.Q8_0.gguf | Q8_0 | 0.09GB |
Original model description:
library_name: transformers tags: []
tinyllamas_92M
<!-- Provide a quick summary of what the model is/does. -->Model Details
max_seq_len = 256
vocab_size = 8192
dim = 768
n_layers = 12
n_heads = 12
n_kv_heads = 12
Training Data
- https://huggingface.co/datasets/roneneldan/TinyStories
- Tokenized using: https://github.com/karpathy/llama2.c?tab=readme-ov-file#custom-tokenizers
Training Hyperparameters
batch_size = 64 # if gradient_accumulation_steps > 1, this is the micro-batch size
dropout = 0.0
# adamw optimizer
gradient_accumulation_steps = 8 # used to simulate larger batch sizes
learning_rate = 1e-3 # max learning rate
max_iters = 34000 # total number of training iterations
weight_decay = 3e-4
beta1 = 0.9
beta2 = 0.95
grad_clip = 1.0 # clip gradients at this value, or disable if == 0.0
# learning rate decay settings
decay_lr = True # whether to decay the learning rate
warmup_iters = 1000 # how many steps to warm up for
Results
4xV100 GPUs used.
Run summary:
iter 34000
loss/train 0.8704
loss/val 0.9966
tokens 983040000
- Created
- Feb 12, 2025
- Updated
- Feb 12, 2025