Downloads · 30 days
12
2% of all-time downloads
SmolLM2-135M-Instruct-bnb-4bit-reasoning-GGUF
Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning-GGUF
SmolLM2-135M-Instruct-bnb-4bit-reasoning-GGUF is a text generation model from Avtrkrb. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
All-time downloads
535
Public
Repo size
863 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf863 MB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Jun 6, 2026
- Updated
- Jun 6, 2026
- SHA
- debb5f17
Try a prompt
Base models
Model card
From the Hugging Face model README
SmolLM2-135M-Instruct-Reasoning-GGUF
GGUF conversions of:
Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning
Available Quantizations
| Quantization | Recommended Use |
|---|---|
| Q4_0 | Same as Q4_K_M |
| Q4_K_M | Best balance of quality and size |
| Q5_K_M | Better quality |
| Q6_K | Better quaity smaller than Q8_0 |
| Q8_0 | Highest quality quantized version |
Chat Template
This model uses the standard SmolLM2 chat template.
<|im_start|>system You are a helpful AI assistant named SmolLM, trained by Hugging Face <|im_end|>
<|im_start|>user Hello <|im_end|>
<|im_start|>assistant
Ollama Example
FROM ./SmolLM2-135M-Instruct-bnb-4bit-reasoning-Q4_K_M.gguf
TEMPLATE """ {{ .Prompt }} """
llama.cpp Example
./llama-cli -m SmolLM2-135M-Instruct-bnb-4bit-reasoning-Q4_K_M.gguf
Notes
When using GGUF versions, ensure the same chat template used during training is applied during inference. Incorrect templates can significantly reduce model quality.
Source Model
Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning
- Task
- Text Generation
- License
- apache-2.0
- Languages
- en
- Created
- Jun 6, 2026
- Updated
- Jun 6, 2026