granite-4.0-micro-GRPO-NuminaMath-20K-GGUF

granite-4.0-micro-GRPO-NuminaMath-20K — GGUF

GGUF quantizations of ermiaazarkhalili/granite-4.0-micro-GRPO-NuminaMath-20K, a GRPO (Group Relative Policy Optimization) reinforcement-learning fine-tune, converted with llama.cpp.

Available quantizations

Recommended default: Q4_K_M. For maximum fidelity use Q8_0.

Usage

llama.cpp

# One-shot
llama-cli -hf ermiaazarkhalili/granite-4.0-micro-GRPO-NuminaMath-20K-GGUF --jinja -p "Your prompt here" -n 256

# Interactive chat
llama-cli -hf ermiaazarkhalili/granite-4.0-micro-GRPO-NuminaMath-20K-GGUF --jinja -cnv

Ollama

ollama run hf.co/ermiaazarkhalili/granite-4.0-micro-GRPO-NuminaMath-20K-GGUF:Q4_K_M

llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="ermiaazarkhalili/granite-4.0-micro-GRPO-NuminaMath-20K-GGUF",
    filename="*q4_k_m.gguf",
    n_ctx=4096,
)
out = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Your prompt here"}],
    max_tokens=256,
)
print(out["choices"][0]["message"]["content"])

Intended use

Research and non-commercial experimentation. This model was RL-tuned on mathematical chain-of-thought reasoning (NuminaMath); it is a reasoning demonstrator, not a general-purpose assistant. Verify outputs before any downstream use.

Limitations

• GGUF quantizations carry unavoidable quality loss relative to the source weights; prefer Q8_0 when fidelity matters.
• Inherits every limitation of the source checkpoint (ermiaazarkhalili/granite-4.0-micro-GRPO-NuminaMath-20K).
• Optimized for mathematical chain-of-thought reasoning (NuminaMath); capability on unrelated tasks is not guaranteed.

Citation

@misc{granite_4_0_micro_grpo_numinamath_20k_gguf,
  author       = {Ermia Azarkhalili},
  title        = {granite-4.0-micro-GRPO-NuminaMath-20K — GGUF quantized},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ermiaazarkhalili/granite-4.0-micro-GRPO-NuminaMath-20K-GGUF}}
}