Downloads · 30 days
49
8% of all-time downloads
DarkKitsune/grug-9b-Q5_K_L-imat-GGUF
grug-9b-Q5_K_L-imat-GGUF is a image-text-to-text model from DarkKitsune. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as mit.
GGUF quantization of ProCreations/grug-9b to Q5KL with importance matrices. Uses Q80 for token embedding and output weights, and Q5KM config for all other weights.
Downloads · 30 days
49
8% of all-time downloads
All-time downloads
644
Public
Repo size
7.1 GB
Likes
1
Public
Click a slice to open those files.
.gguf7.1 GB · 100%
From the Hugging Face model README
GGUF quantization of ProCreations/grug-9b to Q5_K_L with importance matrices. Uses Q8_0 for token embedding and output weights, and Q5_K_M config for all other weights.
Should have a bit higher accuracy than regular Q5_K_M, to help with more complex/long contexts and accuracy in edge cases.
Be warned, the calibration file is mostly English text and code, so performance writing in Chinese may suffer.
grug take big-brain bird model. grug teach bird think small. bird still code good. bird now think 36 token where old bird think 570. this whole model, LoRA already squished inside. download, run, done.
base bird: Ornith-1.0-9B (dense 9B, MIT, very smart, very chatty inside head) grug medicine: LoRA r=64 on grug-think dataset — 100k agent trajectory where all thinking rewritten short grug style. one full epoch, ~140M token, one RTX PRO 6000, ~13 hour.
same harness both birds. same prompt. greedy. no cherry pick. eval code live in dataset repo (scripts/eval_grug.py).
| test | old bird | grug bird | change |
|---|---|---|---|
| HumanEval pass@1 | 91.5% | 79.3% | -12.2 |
| MBPP pass@1 | 80.0% | 78.0% | -2.0 |
| agent replay: tool call valid | 100% | 88.9% | -11.1 |
| agent replay: RIGHT tool picked | 77.8% | 88.9% | +11.1 |
| test | old bird tokens (think part) | grug bird tokens (think part) | change |
|---|---|---|---|
| HumanEval | 958 (570 think) | 249 (36 think) | -74% (-94% think) |
| MBPP | 594 (305 think) | 148 (38 think) | -75% (-88% think) |
| agent replay | 137 (99 think) | 73 (14 think) | -47% (-85% think) |
whole benchmark suite, same GPU, same everything:
| old bird | grug bird | |
|---|---|---|
| wall time | 87.4 min | 26.1 min |
3.3x faster. tokens = time = money. grug bird leave money in pocket.
per SOLVED problem old bird spend ~1047 token. grug bird spend ~314. three grug answer for price of one old answer.
<think>Tuple field likely in fields.py. View file to find implementation.</think>
<think>Need exact line numbers for Tuple class. Use grep.</think>
<think>Understand how _bind_to_schema works in base Field class. Tuple likely
missing override that binds inner fields.</think>
real hypothesis chain. no "Let me carefully analyze". grug already analyzing. grug just do.
from transformers import AutoModelForImageTextToText, AutoTokenizer
import torch
model = AutoModelForImageTextToText.from_pretrained(
"ProCreations/grug-9b", dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("ProCreations/grug-9b")
messages = [{"role": "user", "content": "Write a function that checks if a number is prime."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
need transformers >= 5.8 (qwen3_5 architecture). thinking come in <think> tag. tool calling use same XML <tool_call><function=...> format as base bird. vision tower still inside, untouched, frozen during tune.
grug done. model think small. work same-ish. much cheap. go build.