kuclab-hertz-0.7

KucLab Hertz 0.7

A Czech/English STEM + programming assistant built by KucLab on top of google/gemma-4-12B-it. This release focused on two things: pushing STEM accuracy further, and making identity/personality hold up without needing a system prompt — a real bug found in every prior release (0.3-0.6): ollama pull hf.co/... alone doesn't carry a Modelfile's SYSTEM directive, so the model would answer as generic Gemma unless you separately ran ollama create. See the Quickstart below for the fix, and the "honest development story" section for how deep the identity fix goes.

What this is

Hertz 0.7 is a LoRA fine-tune (r=16, merged into the base weights). The dataset combines Hertz 0.6's full corpus (2600 rows, itself carrying four generations of cumulative Czech/STEM reinforcement) with ~1463 fresh STEM-heavy rows (generated by an autonomous coding-agent subagent, then down-sampled from an initial ~2388 to cut repetitive templated drill problems), 309 CS↔EN terminology rows, and a substantially expanded identity block (56 direct identity rows, up from 15 in 0.6 — including explicit denials for "are you Gemma/GPT/Gemini/Claude" misidentification, and 10 rows where the model's dry/direct personality shows up in ordinary answers, not just identity questions).

• Base: google/gemma-4-12B-it (11.95B params, Apache 2.0)
• Method: QLoRA, r=16 / alpha=32
• Context: 262144 tokens natively. A stretch to ~500k-512k was requested for this release; it is not possible with our current tooling — Gemma-4 stores RoPE settings in a rope_parameters structure with separate config per attention type, and the YaRN scaling technique we use elsewhere is a legacy alias for that same field that overwrites it instead of extending it, breaking GGUF export outright (confirmed the hard way during Hertz 0.6). Context stays at native 262144 this release.
• Training data: 4466 rows total (2600 reused from 0.6, 1463 fresh, 309 terminology, 56 identity, 28 answer-format, 10 personality-infused).

Quickstart (Ollama)

Important: ollama pull hf.co/... alone does NOT apply this model's system prompt — Ollama only fetches the raw GGUF from Hugging Face, it does not read a repo's Modelfile. Use ollama create with the Modelfile below instead — it pulls the weights AND applies the system prompt in one step:

curl -O https://huggingface.co/KucLab/kuclab-hertz-0.7/resolve/main/Modelfile
ollama create kuclab-hertz-0.7 -f Modelfile
ollama run kuclab-hertz-0.7

That said — unlike every prior release, identity now mostly works even without this step, because it's trained directly into the weights this time, not just described in the system prompt. See below.

The identity bug, and how far the fix goes

A user testing Hertz 0.6 via plain ollama pull found it answered "I'm a large language model trained by Google" — because that pull path never applies the Modelfile's system prompt, and 0.6 only had 15 identity training rows (0.5% of its dataset) — not enough signal to override the base model's own strong pretrained identity.

This release trains identity much harder: 56 direct identity rows (up from 15), covering many phrasings and — critically — explicit corrections for "are you Gemma/GPT/Gemini/Claude" style questions, since the base model's prior for those is very strong (it's literally built on Gemma's weights).

Tested without any system prompt at all (the exact scenario that broke 0.6):

• "Kdo jsi?" / "Who are you?" → correctly identifies as KucLab Hertz ✅
• "Jsi Gemma?" → still sometimes reverts to "Ano, jsem Gemma..." ⚠️

So: the default, most-common identity questions are now robust without a system prompt. A specific adversarial "are you Gemma?" question, with only a handful of training examples against a very strong base prior, isn't fully overridden yet. No fine-tuning technique makes behavior 100% unable to be changed by any prompt — this is a real, disclosed, partial improvement, not a complete fix.

Benchmarks — an honest, complicated story

Two full training attempts happened this release, both measured with the same corrected methodology established in Hertz 0.5/0.6 (fixed 1200s timeout, 4096 token budget, identical grading code).

Attempt 1 (5384 rows: full 0.6 reuse + 2388 fresh rows, unfiltered) regressed both benchmarks: MMLU-Pro STEM fell to 68.3% (from 0.6's 79.2%) and Czech terminology fell to 68.0% (from 73.8%), despite the format-compliance win holding (still only 18/240 unparsed, same as 0.6). Diagnosis: 66% of the fresh rows (1582 of 2388) came from just 219 repeated templated computational patterns (only the numbers varied) — heavy narrow drilling that likely crowded out the small LoRA adapter's limited capacity for the broader knowledge MMLU-Pro actually tests, and diluted the terminology/identity signal by nearly doubling total dataset size.

Attempt 2 (this release) down-sampled those templated rows to at most 3 per pattern (2388 → 1463 fresh rows, total dataset 5384 → 4466) and added more identity-denial examples.

MMLU-Pro STEM (240 held-out questions, this project's own curated subset)

Hertz 0.7 lands essentially level with 0.6 (−0.4pp) — not the 85% target for this release, but a large recovery from attempt 1's 68.3%, and the best format-compliance result yet: only 1/240 answers were unparseable (down from 18/240 in both 0.6 and this release's attempt 1), a direct result of the expanded answer-first training.

Czech terminology benchmark (206 held-out CS↔EN scientific terms)

Below Hertz 0.6's peak. Same diagnosis as above (dataset size grew relative to the fixed 309-row terminology block).

Honest status

• ⚠️ MMLU-Pro STEM: 78.8%, essentially level with Hertz 0.6 (79.2%) — the 85% target for this release was not reached
• ⚠️ Czech terminology (69.4%) is below Hertz 0.6's 73.8% — disclosed, not fully recovered despite a second attempt
• ✅ Identity now trained directly into weights, not just the system prompt — default "who are you" questions work with zero system prompt
• ⚠️ Specific adversarial misidentification ("are you Gemma?") is only partially fixed — see above
• ✅ Personality (direct, dry humor, minimal hedging) retained from 0.5/0.6
• ⏳ Context extension beyond native 262144 confirmed architecturally blocked on this base via YaRN — not solved this release
• ⏳ No tool-calling fine-tuning; no uncensoring pass beyond the personality shift already in 0.5/0.6

License

Apache 2.0, inherited from google/gemma-4-12B-it (per Google's official Hugging Face listing).

Credits

• Base model: google/gemma-4-12B-it (Google, Apache 2.0)
• Fine-tuning, dataset construction, and packaging: KucLab