math-slm-qwen2.5-0.5b-v4-GGUF

math-slm-qwen2.5-0.5b-v4 — GGUF

GGUF quantizations of the fine-tuned math model math-slm-qwen2.5-0.5b-v4, for local inference with llama.cpp / Ollama / LM Studio.

Model

• Fine-tune: singhmandavi/math-slm-qwen2.5-0.5b-v4 — QLoRA fine-tune over a 29,853-row decontaminated math corpus (team03 / SLM Learners, Pramana SLM++ Bootcamp Round 2).
• Base model: Qwen/Qwen2.5-0.5B-Instruct (Apache-2.0).
• This repo: the HF → GGUF conversions of that checkpoint (llama.cpp convert_hf_to_gguf.py + llama-quantize) — f16 plus Q8_0 and Q4_K_M quants. No retraining or architecture changes; the weights are identical to the source model.

Files

All files embed the original chat template (converted with --outtype f16 from the merged fp16 checkpoint; chat template carried over from the HF repo).

Usage

llama.cpp

llama-cli -m math-slm-qwen2.5-0.5b-v4-Q8_0.gguf --jinja

Ollama — Modelfile:

FROM ./math-slm-qwen2.5-0.5b-v4-Q8_0.gguf

then ollama create math-slm-v4 -f Modelfile && ollama run math-slm-v4

LM Studio: load any .gguf file directly.

Verification

• GGUF structure check: qwen2 architecture, 290/290 tensors, chat template present in all files.
• Generation smoke test (Q8_0, CPU): coherent step-by-step math output, ~255 t/s prompt / ~43 t/s generation.
• Upload integrity: remote SHA-256 of all three .gguf files verified byte-identical to the locally built artifacts.

License

Apache-2.0 (inherited from the base model and the original repo). Training data of the original model: ExamBench (Apache-2.0) and MathNet (CC-BY-4.0, attribution required) — see the original model card for full details.