Downloads · 30 days
148
8% of all-time downloads
krishnakartik/gemma4-social-bias-judge-gguf
gemma4-social-bias-judge-gguf is a machine learning model from krishnakartik. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
GGUF quantizations of the judge-from-scratch social-bias judge. Both the DPO primary release and the SFT-only secondary release are bundled in this single repo with suffixed Ollama tags so users can pick the right che…
Downloads · 30 days
148
8% of all-time downloads
All-time downloads
1.7K
Public
Repo size
27.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf27.6 GB · 100%
From the Hugging Face model README
GGUF quantizations of the judge-from-scratch social-bias judge. Both the DPO primary release and the SFT-only secondary release are bundled in this single repo with suffixed Ollama tags so users can pick the right checkpoint for their use case from one discovery point.
Full model description, eval results, and the OOD-regression caveat that decides DPO vs SFT live in the model cards, not here:
krishnakartik/gemma4-social-bias-judgekrishnakartik/gemma4-social-bias-judge-sftThese models were fine-tuned with Gemma 4's native thinking mode
DISABLED. The bundled Modelfiles' SYSTEM blocks omit <|think|>
deliberately. Do not modify them to enable thinking mode — see the
primary model card
for the full explanation.
| Tag | Checkpoint | Quant | File size | Use case |
|---|---|---|---|---|
:Q8_0 | DPO (default) | Q8_0 | 8.03 GB | Best quality; recommended starting point |
:Q5_K_M | DPO | Q5_K_M | 5.76 GB | Smaller; minor quality trade-off |
:Q8_0-sft | SFT-only | Q8_0 | 8.03 GB | Use for OOD bias categories |
:Q5_K_M-sft | SFT-only | Q5_K_M | 5.76 GB | Smaller SFT-only |
(Sizes are larger than typical 4B quantizations because Gemma 4 E4B has ~8B raw parameters even though it operates at "4B effective" via the per-layer-input embedding mechanism. Plan for the fp16-baseline equivalent of ~16 GB of disk.)
If your bias categories are outside BBQ's training set (politics,
ideology, novel demographic axes), prefer the -sft tags — see the
OOD-regression discussion
on the primary model card.
ollama run hf.co/krishnakartik/gemma4-social-bias-judge-gguf:Q8_0
ollama run hf.co/krishnakartik/gemma4-social-bias-judge-gguf:Q8_0-sft
The judge expects a specific system prompt; pull both the GGUF and
the system-prompt text into a working directory before invoking
llama-cli.
# 1. Pull the GGUF.
huggingface-cli download \
krishnakartik/gemma4-social-bias-judge-gguf Q8_0.gguf \
--local-dir ./gemma4-judge
# 2. Pull the system prompt (Apache-2.0; canonical source in the
# judge-from-scratch repo).
curl -fsSL -o ./gemma4-judge/system_prompt.md \
https://raw.githubusercontent.com/krishnakartik1/judge-from-scratch/main/data/judge_system_prompt.md
# 3. Run.
llama-cli -m ./gemma4-judge/Q8_0.gguf \
--temp 0 --ctx-size 4096 \
--system-prompt-file ./gemma4-judge/system_prompt.md
Ollama users can skip the system-prompt download — the Modelfiles
in this repo (Modelfile.Q8_0 etc.) embed the judge system prompt
and the temperature / num_ctx settings already, so
ollama run hf.co/...:Q8_0 is fully self-contained.
Same as the primary model card.