kuclab-hertz-0.3

KucLab Hertz 0.3

A Czech/English STEM assistant built by KucLab on top of Qwen2.5-14B-Instruct, fine-tuned to be sharper on physics, chemistry, biology and mathematics in Czech — while carrying forward the base model's general capability, 128k-token context reach, and native tool-calling support.

What this is

Hertz 0.3 is a LoRA fine-tune (r=8, merged into the base weights) trained on a self-distilled corpus of Czech/English STEM concepts, worked problems, and formatting examples. Training data was generated by prompting Qwen3.8-27B (Alibaba/Tongyi, Apache 2.0) — a stronger reasoning model used purely as a data-generation teacher, never trained itself — and used to fine-tune the actual 14B model that ships here.

• Base: Qwen/Qwen2.5-14B-Instruct (14.7B params, Apache 2.0)
• Method: QLoRA, r=8 / alpha=16, merged to fp16 then quantized
• Context: extended to 128k via YaRN rope scaling (native 32k → 131072, factor 4.0)
• Training data: 242 self-distilled rows — Czech/English STEM concepts & terminology, worked problems with step-by-step reasoning, Markdown/code formatting examples, identity
• Format available: GGUF (q4_k_m, ~9GB) for llama.cpp/Ollama, plus the raw LoRA adapter for anyone who wants to merge it onto their own copy of the base model

Why a fine-tune instead of a bigger model

Qwen2.5-14B fits comfortably on a single consumer/cloud GPU (23GB) for both inference and QLoRA training. Rather than chase parameter count, this project focuses the model's existing capacity on a specific gap: strong, natural Czech STEM terminology and step-by-step worked solutions, which the base model handles less fluently out of the box.

Quickstart (Ollama)

Important: ollama pull hf.co/... alone does NOT apply this model's system prompt (identity + personality) — Ollama only fetches the raw GGUF from Hugging Face, it does not read a repo's Modelfile. Without the system prompt, the model falls back to identifying as a generic Gemma model. Use ollama create with the Modelfile below instead — it pulls the weights AND applies the system prompt in one step:

curl -O https://huggingface.co/KucLab/kuclab-hertz-0.3/resolve/main/Modelfile
ollama create kuclab-hertz-0.3 -f Modelfile
ollama run kuclab-hertz-0.3

(The Modelfile's FROM line points at hf.co/KucLab/kuclab-hertz-0.3:Q4_K_M, so this pulls the same GGUF automatically — no separate download needed.)

Or build from the Modelfile in this repo with the bundled GGUF.

Example

Q: Kolik je 17 × 23? Ukaž postup.

A: 17 × 23 = 17 × (20 + 3) = 340 + 51 = 391

Q: Vysvětli druhý Newtonův zákon.

A: (explains F = dp/dt, in fluent Czech, with the formula and its meaning)

Benchmarks

Measured against the same base model (qwen2.5:14b-instruct, Q4_K_M via Ollama), same prompts, same grading code, both models cold.

MMLU-Pro STEM (240 held-out questions, this project's own curated subset — not a claim of parity with the standard published MMLU-Pro leaderboard number)

Czech scientific terminology (206 held-out terms, both translation directions, this project's own curated set)

Read this as: no regression on general STEM reasoning, a real (if modest) gain on Czech terminology — which is exactly what the fine-tune targeted. Physics moved the most; biology and math moved slightly the other way. These are two custom benchmarks built for this project, not standardized public leaderboards — useful for before/after comparison on this exact model, not for cross-model bragging rights.

Honest status

This is a small-scale, single-GPU fine-tuning project, not a frontier lab release. What's verified:

• ✅ Coherent, correct Czech and English output (spot-checked: arithmetic, physics, chemistry, terminology)
• ✅ Correctly identifies as a KucLab model, not as "Qwen" (LoRA + system prompt)
• ✅ 128k context window configured and loadable (long-context quality not yet independently verified with held-out long documents)
• ✅ MMLU-Pro STEM and Czech terminology benchmarked against base (see above) — no regression, modest gains

What's not done yet:

• ⏳ Tool-calling fine-tuning — the base model supports function calling natively, but this fine-tune did not add tool-use training examples
• ⏳ No uncensoring/decensoring pass — safety behavior is inherited from the base model as-is
• ⏳ No standardized public benchmarks (GSM8K, HumanEval, official MMLU, etc.) — only the two project-specific evals above

If you're deciding whether this fits your use case: treat it as "Qwen2.5-14B-Instruct, nudged toward Czech STEM fluency and KucLab identity," not as a categorically different or more powerful model than its base.

License

Apache 2.0, inherited from Qwen2.5-14B-Instruct. This repository includes a LoRA adapter and a merged/quantized derivative of that base model; see LICENSE for full terms. Qwen3.8-27B (used only to generate training data, never trained or redistributed here) is separately licensed under Apache 2.0 by Alibaba/Tongyi.

Credits

• Base model: Qwen/Qwen2.5-14B-Instruct (Alibaba Cloud, Apache 2.0)
• Training-data teacher: Qwen/Qwen3.8-27B (Alibaba/Tongyi, Apache 2.0)
• Fine-tuning, dataset construction, and packaging: KucLab