Smolpus-3B-GGUF-Q4

Smolpus-3B (GGUF Q4_K_M)

A fine-tuned version of SmolLM3-3B trained on a distillation dataset generated from a proprietary frontier model, quantized to Q4_K_M GGUF for local inference.

File: SmolLM3-3B-4bit.Q4_K_M.gguf — 1.92 GB

Training Details

• Base model: HuggingFaceTB/SmolLM3-3B
• Fine-tuning method: Supervised Fine-Tuning (SFT)
• Dataset: Private distillation dataset; not publicly available
• Hardware: Apple M5 Max (128GB unified memory)
• Framework: MLX / Unsloth Studio

Usage

llama.cpp

llama serve -hf Spakie/Smolpus-3B-GGUF-Q4:Q4_K_M

Ollama

ollama run hf.co/Spakie/Smolpus-3B-GGUF-Q4:Q4_K_M

llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="Spakie/Smolpus-3B-GGUF-Q4",
    filename="SmolLM3-3B-4bit.Q4_K_M.gguf",
)
llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain what a transformer is."}]
)

Performance

No evals run yet on this fine-tune. Refer to the base model card for SmolLM3-3B benchmark results (may be innaccurate).

Limitations

• May reflect stylistic patterns from the distillation source
• Not independently evaluated for safety or bias beyond the base model
• Generated content may be factually inaccurate; verify important outputs

License

Apache 2.0 (inherited from base model). Fine-tuning dataset is not released.