mochi-GGUF

🍡 Mochi GGUF

GGUF quantizations of artindnr/mochi, a math-reasoning fine-tune of GLM-4.7-Flash trained on the Open Math Reasoning (mini) dataset — the same chain-of-thought data behind the winning submission to the AIMO-2 Kaggle competition.

These quants let you run Mochi locally with llama.cpp, Ollama, LM Studio, or any other GGUF-compatible runtime.

Files

If you're unsure which to pick: Q5_K_M is a solid default. Use Q8_0 if you have the VRAM/RAM to spare and want maximum fidelity, and Q4_K_M if you're constrained on memory or want faster inference.

Usage

llama.cpp

./llama-cli -m mochi.Q5_K_M.gguf -p "If x^2 - 5x + 6 = 0, what are the values of x?" -n 512

Or serve it as an OpenAI-compatible endpoint:

./llama-server -m mochi.Q5_K_M.gguf -c 4096

Ollama

Create a Modelfile:

FROM ./mochi.Q5_K_M.gguf

Then:

ollama create mochi -f Modelfile
ollama run mochi

LM Studio

Download the .gguf file of your choice directly in LM Studio's model browser (search artindnr/mochi-gguf), or drop the file into your local models folder.

About Mochi

Mochi is a chain-of-thought math fine-tune of GLM-4.7-Flash. See the full model card for training details, dataset info, and intended use.

• Base model: GLM-4.7-Flash
• Fine-tuning data: unsloth/OpenMathReasoning-mini
• Focus: Step-by-step mathematical reasoning (olympiad-style problems)
• Format: GGUF, for use with llama.cpp and compatible runtimes

Limitations

• Quantization introduces some precision loss versus the original fp16/bf16 weights — expect small quality differences between Q8_0, Q5_K_M, and Q4_K_M, especially on harder problems.
• Fine-tuned specifically for math reasoning; general chat ability may differ from the base GLM-4.7-Flash model.
• Always verify important results — this is not a substitute for a calculator or formal proof checker.

Acknowledgements

• GLM-4.7-Flash for the base model
• Unsloth for fine-tuning tooling
• NVIDIA's AIMO-2 team for the OpenMathReasoning dataset
• llama.cpp for the GGUF format and quantization tooling