gemma-4-31B-caveman

gemma-4-31B-caveman

Gemma 4 31B fine-tuned to speak caveman-mode natively. Full bf16 weights, drop-in replacement for google/gemma-4-31B-it.

Drops articles, filler, pleasantries, hedging. Allows fragments. Keeps code blocks, function names, error strings, and CLI commands byte-exact. Pattern: [thing] [action] [reason]. [next step].

For the lighter LoRA-adapter variant see JBrussee/gemma-4-31B-caveman-lora.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("JBrussee/gemma-4-31B-caveman")
model = AutoModelForCausalLM.from_pretrained(
    "JBrussee/gemma-4-31B-caveman",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

msgs = [{"role": "user", "content": "Explain database connection pooling."}]
ids = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True).to(model.device)
out = model.generate(ids, max_new_tokens=300, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Example

Prompt: "Why does my React component re-render every time the parent updates?"

Output:

Parent re-render → child re-render by default. Props "change" each render
if inline obj/array/fn → new ref → child sees new props.

Fix:
- Wrap child: React.memo(Child)
- Stabilize props: useMemo for obj/array, useCallback for fn
- Lift state so parent re-render not needed

Check: if child memo'd but still re-renders, inline prop is culprit.

Training

• Method: QLoRA NF4 + double-quant + bf16 compute (Unsloth), then merged to bf16
• LoRA: rank 16, α 32, dropout 0, targets q,k,v,o,gate,up,down
• Loss: assistant-only loss masking via TRL completion_only_loss=True
• Epochs: 3, lr 2e-4 cosine, batch 2 × grad-accum 8 (effective 16)
• Hardware: RunPod RTX PRO 6000 Blackwell 96 GB
• Time: ~50 minutes total

Data

1750 train + 193 eval pairs, source-normal → caveman style transfer, drawn from six permissive datasets:

Caveman side synthesized via Claude Code (claude -p) and Codex CLI (codex exec with GPT-5.5), routed through the SKILL.md ruleset.

Eval (n=193 holdout)

Final train_loss 0.024 · eval_loss 0.72 · eval_mean_token_accuracy 0.815.

Strengths: code preservation (96-100% fence-exact), low article density (0.5-2%), strong semantic preservation (91-98%). Weakness: compression ~10-40% rather than the 50-70% of the gold caveman pairs. To compress harder, retrain on a tighter-filtered dataset (filter upper bound 0.7 instead of 1.0).

Reproduce

Full code, data pipeline, and configs: https://github.com/JuliusBrussee/finetune-caveman

License

Inherits the Gemma Prohibited Use Policy. Apache 2.0 base + Gemma terms apply to all outputs. Repository code is MIT. The caveman style ruleset is MIT (https://github.com/JuliusBrussee/caveman).

Citing

@misc{brussee2026cavemanGemma,
  author = {Julius Brussee},
  title  = {Caveman-mode Gemma 4 31B},
  year   = {2026},
  url    = {https://huggingface.co/JBrussee/gemma-4-31B-caveman}
}