grug-3b

grug-3b

small grug. think in grug, answer in normal english. never other way round.

base model Nanbeige4.2-3B is smart but think A LOT. ask it capital of france and it burn 482 think token on it. that is where grug find free lunch.

grug fix the thing people complain about

old grug (27b, 35b) think short. ALWAYS short. easy question short - good. hard question also short - bad, grug not do the work, answer come out worse. people tell grug this. grug listen.

grug-3b think short when problem small, long when problem big (median think token):

grug hard/easy think ratio 57x (median). base only 6x - base already near ceiling on easy question, cannot stretch. grug start from almost nothing and stretch when problem earn it.

the scoreboard - honest one

matched subset, same problems both model, greedy where noted:

grug beat base on hard math (MATH-500 +10) and on factual multiple choice (ARC +12.5). base overthink those and talk itself out of right answer. grug check work instead of spiral.

grug lose on HumanEval and MBPP and GSM8K. that is real cost, grug not hide it. squeeze 48% of think token out of 3B brain, some code and some arithmetic fall out. if you need max code pass rate and token cheap, use base. if you want most of the brain at fraction of token, grug good trade.

how grug learn to check work

first grug-3b lost GSM8K bad (81.2 vs 96.2). grug look at what break: think say "204 + 160 + 330" then write 710. gold 694. numbers all correct, sum wrong. grug never re-read own sum.

making think LONGER did not fix it (81.2 at both 86 and 107 think token). what fix it: mandatory second-pass arithmetic check in every numeric think, ~10 word cost:

Check: 204+160+330 -> 330+160=490, +204=694 ok.

that single rule took GSM8K 81.2 -> 86.2, MATH-500 64.0 -> 76.0, MBPP 76.0 -> 82.0. cheap habit, big brain.

the one rule

think is grug. answer is NEVER grug.

Alicia n candies, 3-digit. n+5 = 0 mod 8 -> n = 3 mod 8. n-8 = 0 mod 5 -> n = 3 mod 5.
gcd(5,8)=1, CRT -> n = 3 mod 40. n = 40k+3. smallest 3-digit: k=3 -> 123.
check: 128/8=16 ok, 115/5=23 ok. k=2 gives 83 < 100. answer 123.

answer come out clean english, full sentence, markdown, latex. measured: grug answer function-word ratio 0.3071 vs base 0.2135 - grug answer read MORE like normal english than base answer do. user never see grug unless user open think tag.

what grug-3b is

• base: Nanbeige/Nanbeige4.2-3B (apache 2.0)
• method: LoRA r=32 on all attention + mlp linears, merged to bf16
• data: grug-3b-train - 4,524 rows, difficulty-tiered fresh generation (math, code, facts, identity)
  • proven grug agentic data. tier decide think budget, validator enforce it.
• general on purpose: math, facts, code, tool use. not agent-coding-only like grug-27b.

known rough edge

on medium/hard problem grug sometimes run long and not close </think> inside 6144 token (91.7% closed at medium, 75.0% at very hard; base 66.7% at very hard). median stay small (208 and 737 token) - it is a tail, not the middle. give grug room (max_new_tokens>=2048) on hard problem.

use grug

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("ProCreations/grug-3b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("ProCreations/grug-3b",
    trust_remote_code=True, dtype=torch.bfloat16, device_map="auto")

msgs = [{"role": "user", "content": "Solve x^3 - 3x^2 + 2x = 0."}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=2048)
print(tok.decode(out[0], skip_special_tokens=True))

template open <think> already. grug fill it, close it, then talk normal.

transformers >= 4.54 fix, shipped in this repo

upstream modeling_nanbeige.py call Cache.get_max_length(), removed in 4.54. replacement get_max_cache_shape() report -1 for unbounded cache where old contract say None. pass -1 through and attention_mask get cropped one token, position_ids go one short of hidden states, RoPE blow up on first forward.

grug ship patched modeling_nanbeige.py here. no user shim needed. fix verified by checking cached generation match uncached generation exactly.

gguf

• ProCreations/grug-3b-gguf - f16 + Q8_0/Q6_K/Q5_K_M/Q4_K_M/Q3_K_M
• ProCreations/grug-3b-qat-q4-gguf - q4 trained to survive squeeze

llama.cpp need nanbeige support - PR ggml-org/llama.cpp#25994, branch Nanbeige/llama.cpp:nanbeige42, until it merge upstream.

grug family

grug-9b, grug-27b, grug-35b, grug-3b. all open. apache 2.0.

grug made by ProCreations. grug not made by openai. grug not made by google. grug not official nanbeige product. grug just grug.