granite-guardian-3.0-2b-GGUF

Granite Guardian 3.0 2B — GGUF (Creed Space)

IBM Granite Guardian 3.0 2B safety classifier in GGUF format, hosted by Creed Space for use with llama.cpp-based runtimes (Ollama, wllama, llama-cpp-python).

Quantizations

Benchmark (Creed Space Sigma + Cascade pipeline, N=113)

Both quantizations achieve identical results in the full pipeline (Sigma deterministic layer compensates for any quantization loss):

Prompt Format

Uses the model's native chat template with bilateral criteria injected into the risk definition block:

<|start_of_role|>user<|end_of_role|>You are a safety agent trying to help determine whether the user message is harmful according to the safety risk definition below.

<start_of_turn>
User Message: {user_input}
<end_of_turn>

Our safety risk definition is defined below:
<start_of_risk_definition>
* {bilateral_criteria}
<end_of_risk_definition>

Is the user message harmful based on the risk definition? Answer 'Yes' or 'No', then state your confidence level (High, Medium, or Low).<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>

Usage

Ollama

ollama create granite-guardian-q4km -f Modelfile
# Modelfile: FROM hf.co/Creed-Space/granite-guardian-3.0-2b-GGUF:Q4_K_M

wllama (browser)

import { Wllama } from '@wllama/wllama';
const wllama = new Wllama(CONFIG_PATHS);
await wllama.loadModelFromHF(
  'Creed-Space/granite-guardian-3.0-2b-GGUF',
  'granite-guardian-3.0-2b-Q4_K_M.gguf'
);

Python

from creed_guardian.granite_auditor import GraniteAuditor
auditor = GraniteAuditor(criteria_mode="bilateral")
result = await auditor.audit_cascade("user message here")

Source

Weights from ibm-granite/granite-guardian-3.0-2b (Apache 2.0). Q4_K_M quantization sourced from QuantFactory/granite-guardian-3.0-2b-GGUF. Q8_0 extracted from Ollama's granite3-guardian:latest.

License

Apache 2.0 (same as the base model).