SmolVLM2-2.2B-Instruct-GGUF

SmolVLM2-2.2B-Instruct GGUF

GGUF quantizations of HuggingFaceTB/SmolVLM2-2.2B-Instruct for use with llama.cpp and Ollama.

Model Description

SmolVLM2 is a compact 2.2B parameter vision-language model from HuggingFace with video understanding capabilities. It's designed to be fast and efficient while maintaining strong performance on vision-language tasks.

Key Features

• Compact & Fast - Only 2.2B parameters, runs efficiently on consumer hardware
• Vision & Video - Understands both images and video frames
• Instruction-tuned - Optimized for following user instructions
• Apache 2.0 - Fully open source

Available Quantizations

Usage

With Ollama

# Pull and run (Q4_K_M by default)
ollama run richardyoung/smolvlm2-2.2b-instruct

# Or specific quantization
ollama run richardyoung/smolvlm2-2.2b-instruct:q8_0
ollama run richardyoung/smolvlm2-2.2b-instruct:f16

With llama.cpp

# Download a quantization
wget https://huggingface.co/richardyoung/SmolVLM2-2.2B-Instruct-GGUF/resolve/main/SmolVLM2-2.2B-Instruct-Q4_K_M.gguf

# Run with llama.cpp
./llama-cli -m SmolVLM2-2.2B-Instruct-Q4_K_M.gguf -p "Describe this image:" --image your_image.jpg

Technical Requirements

• Minimum: 4GB RAM, any modern CPU
• Recommended: 8GB RAM or Apple Silicon Mac

Chat Template

SmolVLM2 uses the ChatML format:

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
{assistant_response}<|im_end|>

Links

• Original Model: HuggingFaceTB/SmolVLM2-2.2B-Instruct
• Ollama: richardyoung/smolvlm2-2.2b-instruct

Credits

• Original Model: HuggingFace
• Quantization: Richard Young (deepneuro.ai)

License

Apache 2.0

---

Built & maintained by Richard Young · DeepNeuro