Downloads · 30 days
319
9% of all-time downloads
SmolLM3-3B-GGUF
Bastion-AI/SmolLM3-3B-GGUF
SmolLM3-3B-GGUF is a machine learning model from Bastion-AI. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
All-time downloads
3.5K
Public
Repo size
22.1 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf27.8 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jul 23, 2025
- Updated
- Jul 23, 2025
- SHA
- 8729c113
Base models
Model card
From the Hugging Face model README
SmolLM3-3B GGUF Quantized Models
📌 Original Model Reference
This repository provides quantized GGUF versions of the original model:
All quantizations are derived from this base model. Please refer to the original model card for architecture, training details, and upstream updates.
🧩 Model Description
This repository contains multiple quantized GGUF versions of the SmolLM3-3B model, converted for efficient inference on CPU and mobile devices.
- Quantized by: Bastion-AI
- BastionAI Website: https://bastionai.github.io/
- Format: GGUF (for llama.cpp and compatible)
BastionAI empowers users with advanced AI capabilities that run entirely on your device. Learn more about our privacy-first, local AI solutions at bastionai.github.io.
🚀 How to Run Locally
You can use these GGUF files with:
Example command:
./main -m SmolLM3-3B.Q4_K_M.gguf -p "Explain gravity in simple terms."
📦 Quantized GGUF Downloads
| Quant Type | Disk Size | Download Link | Details |
|---|---|---|---|
| Q2_K | 1.2G | Download | Standard 2-bit quant |
| Q3_K | 1.5G | Download | Standard 3-bit quant |
| Q3_K_S | 1.3G | Download | Small 3-bit quant |
| Q3_K_M | 1.5G | Download | Medium 3-bit quant |
| Q4_0 | 1.7G | Download | Standard 4-bit quant |
| Q4_K | 1.8G | Download | Standard 4-bit quant |
| Q4_K_S | 1.7G | Download | Small 4-bit quant |
| Q4_K_M | 1.8G | Download | Medium 4-bit quant |
| Q5_0 | 2.0G | Download | Standard 5-bit quant |
| Q5_K | 2.1G | Download | Standard 5-bit quant |
| Q5_K_S | 2.0G | Download | Small 5-bit quant |
| Q5_K_M | 2.1G | Download | Medium 5-bit quant |
| Q6_K | 2.4G | Download | Standard 6-bit quant |
| Q8_0 | 3.0G | Download | Standard 8-bit quant |
🛡️ About BastionAI
BastionAI empowers users with advanced AI capabilities that run entirely on your device. Experience the full potential of AI conversation, document processing, and voice interaction without compromising your privacy or security.
- Website: https://bastionai.github.io/
- Products: BastionChat, BastionSDK
- Mission: Democratizing AI for everyone, privacy-first, open source, cross-platform.
📖 License
This model is distributed under the Apache 2.0 license, as per the original HuggingFaceTB/SmolLM3-3B.
🙏 Credits
- Original model by HuggingFaceTB
- GGUF conversion and quantization by Bastion-AI
- Learn more about BastionAI: https://bastionai.github.io/
🔗 Links
📣 Community & Support
📜 Citation
If you use this model, please cite the original SmolLM3-3B and BastionAI as appropriate.
@misc{smollm3,
title={SmolLM3-3B},
author={HuggingFaceTB},
year={2024},
url={https://huggingface.co/HuggingFaceTB/SmolLM3-3B}
}
- License
- apache-2.0
- Created
- Jul 23, 2025
- Updated
- Jul 23, 2025