SmolLM3-3B-Q4_K_M-GGUF

ShayonSarker/SmolLM3-3B-Q4_K_M-GGUF

SmolLM3-3B-Q4_K_M-GGUF is a text generation model from ShayonSarker. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.

Downloads · 30 days

186

39% of all-time downloads

All-time downloads

475

Public

Repo size

5.2 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf5.2 GB · 100%

At a glance

Task: Text Generation
License: apache-2.0
Access: Public
Created: Sep 15, 2026
Updated: Sep 26, 2026
SHA: ce8811bc

Try a prompt

Base models

HuggingFaceTB/SmolLM3-3B-Base

Model card

From the Hugging Face model README

SmolLM3-3B-Base Q4_K_M GGUF — verified 4GB edge build

Downloads: Hugging Face — ShayonSarker/SmolLM3-3B-Q4_K_M-GGUF · Source & build recipes: GitHub — edge-smollm3-gguf · Base model: HuggingFaceTB/SmolLM3-3B-Base

The GGUF binaries are hosted on Hugging Face; this repository holds the reproducible conversion pipeline, calibration data, and checksums.

Base: HuggingFaceTB/SmolLM3-3B-Base (SmolLM3ForCausalLM, Apache-2.0) · Quant: llama.cpp 0.4.1-dev (c77ae69) · imatrix on 4000-line wikitext-2 (seed 42, 798 chunks, 4 threads)

Format status

File	Status	Note
`smol-Q4_K_M.gguf`	Published	Main release, ~1.8 GB
`smol-Q8_0.gguf`	Published	~3.1 GB
F16	Reference only	5.8 GB source-side reference used for the PPL baseline; not published

The F16 row in the eval table below is a measurement reference, not a downloadable file.

Eval (wikitext, 128 held-out lines, 2048 ctx)

model	PPL	size
F16	6.3582	5.8 GB
Q8_0	6.3667 (+0.14%)	3.1 GB
Q4_K_M	6.5020 (+2.26%)	1.8 GB

Gate was <5% — passed. Task evals (ARC/HellaSwag) not run: perplexity only, stated honestly. Files: smol-Q4_K_M.gguf, smol-Q8_0.gguf, imatrix.dat, calib.txt, REPRO.json, SHA256.txt (see manifest). Full build log + repro scripts: edge-smollm3-gguf on GitHub.

Verified outputs

Verbatim smol-Q4_K_M.gguf completions, --temp 0 -no-cnv, 24 new tokens, prompt form Question: ...\nAnswer:. The model answers, then continues with further Question:/Answer: pairs of its own invention; those are trimmed.

Question	Model answer
What is the capital of Japan?	`Tokyo is the capital of Japan.`
What is the capital of Italy?	`Rome is the capital of Italy.`
What is the capital of Egypt?	`Cairo is the capital of Egypt. It is the largest city in Egypt and the Arab world.`
What is the largest ocean on Earth?	`The Pacific Ocean is the largest ocean on Earth. It covers about 30 percent of the Earth's surface`
Which planet is closest to the Sun?	`Mercury is the closest planet to the Sun. It is the smallest planet in the Solar System, with a diameter of`
How many days are in a leap year?	`366 days`
How many continents are there?	`There are seven continents: Asia, Africa, North America, South America, Antarctica, Europe, and Australia.`
What is the chemical symbol for gold?	`Au. The symbol for gold is Au, which comes from the Latin word for gold, aurum.`

Eight of eight are correct. This is the lowest-perplexity model in this release set (6.36 F16 on wikitext vs 11.50 for Qwen3-4B), though wikitext perplexity is not a reasoning score and says nothing about multilingual or instruction behaviour.

Use

llama.cpp:

./llama-cli -m smol-Q4_K_M.gguf -p "Explain overfitting in 3 lines" -c 4096

Ollama (see Modelfile):

ollama create smollm3-3b-base-q4 -f Modelfile
ollama run smollm3-3b-base-q4 "Explain overfitting in 3 lines"

4GB GTX 1650: Q4_K_M peaks ~3.4GB, fully offline.

Limitations

Base (non-instruct) model: raw completion, no chat tuning. Prompt accordingly.
English wikitext calibration: quality on other languages/domains may vary.
Perplexity-only validation; verify on your task before production use.

Task: Text Generation
License: apache-2.0
Created: Sep 15, 2026
Updated: Sep 26, 2026