Downloads · 30 days
17
3% of all-time downloads
TinyLlama-1.1B-Chat-GGUF
Arivukkarasu/TinyLlama-1.1B-Chat-GGUF
TinyLlama-1.1B-Chat-GGUF is a machine learning model from Arivukkarasu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as mit.
This is a quantized GGUF version of TinyLlama/TinyLlama-1.1B-Chat, quantized to Q4KM using llama.cpp's quantization tool.
All-time downloads
519
Public
Repo size
668 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf668 MB · 100%
At a glance
- Library
- llama.cpp
- License
- mit
- Access
- Public
- Created
- Jul 14, 2025
- Updated
- Jul 15, 2025
- SHA
- 06f36326
Base models
Model card
From the Hugging Face model README
TinyLlama-1.1B-Chat - Q4_K_M GGUF
This is a quantized GGUF version of TinyLlama/TinyLlama-1.1B-Chat, quantized to Q4_K_M using llama.cpp's quantization tool.
Model Info
- Base model: TinyLlama-1.1B-Chat
- Quantization: Q4_K_M
- Format: GGUF (compatible with llama.cpp and llama-cpp-python)
- Size: ~500 MB
- Use case: Efficient CPU inference for chat and assistant use-cases
📦 Files
TinyLlama-1.1B-Chat-Q4_K_M.gguf: The quantized model
How to Use (with llama-cpp-python)
from llama_cpp import Llama
llm = Llama(model_path="TinyLlama-1.1B-Chat-Q4_K_M.gguf")
output = llm("Who are you?", max_tokens=128)
print(output)
Credits
- Original model: TinyLlama/TinyLlama-1.1B-Chat
- Quantized with: llama.cpp
- Library
- llama.cpp
- License
- mit
- Created
- Jul 14, 2025
- Updated
- Jul 15, 2025