khayyam-math-qwen2.5-7b-v5.1

Khayyam Math — Qwen 2.5-7B v5.1 (LoRA adapter)

A LoRA fine-tune of Qwen2.5-7B-Instruct specialised for generating deterministic SVG figures and learner-facing narrations for math education. v5.1 is the first release trained on production telemetry from khayyammath.com — 52 anonymised turns layered on top of the 2,350-example synthetic teacher corpus.

v5.1 ties v4 on the 20-prompt held-out practical-test battery (20 / 20 vs 20 / 20, zero regressions). The qualitative jump over v4 will need a richer eval to demonstrate; v5.1 is the v4 floor + a process upgrade (production telemetry now reaches training).

• 🌐 Live demo: khayyammath.com
• 📦 Source code & package: github.com/khayyam-math/khayyam-math
• 📊 Benchmark: khayyam-math/lean-math-1000 (coming)

⚠️ Use khayyam_math >= 0.4.1 to load this model. Earlier releases of the package have a JSON-extractor bug that mis-handles roughly 15 % of v5.1's structured outputs (model emits JSON-escaped SVG that breaks the old fallback path). v0.4.1 ships the unescape fallback.

---

Quick start (with the khayyam-math package, ≥ 0.4.1)

pip install "khayyam-math[qwen] @ git+https://github.com/khayyam-math/khayyam-math"

from khayyam_math import KhayyamMath

client = KhayyamMath(
    provider="qwen",
    model="khayyam-math/khayyam-math-qwen2.5-7b-v5.1",
)
result = client.generate("Solve x^2 - 5x + 6 = 0")
print(result.svg)

Quick start (raw transformers + peft)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base    = "Qwen/Qwen2.5-7B-Instruct"
adapter = "khayyam-math/khayyam-math-qwen2.5-7b-v5.1"

tokenizer = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained(
    base, torch_dtype=torch.bfloat16, device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

Serving with vLLM

vllm serve Qwen/Qwen2.5-7B-Instruct \
  --enable-lora \
  --lora-modules khayyam-v5.1=khayyam-math/khayyam-math-qwen2.5-7b-v5.1 \
  --max-lora-rank 8 --dtype bfloat16

(Note --max-lora-rank 8, halved from v4's 16.)

---

Training

Why v5.1 and not v5

v5 (rank 16, alpha 32, 3 epochs) over-fit: training loss collapsed to 0.048, and at inference the model emitted truncated or empty SVG on the most element-dense prompts and started breaking JSON structure across raw newlines. v5 failed the practical test (15 / 20) and was archived as qwen_lora_v5_rejected/. v5.1 keeps the same corpus but halves LoRA capacity (rank 8) and drops one epoch — the "less is more" pattern PEFT papers recommend for small-corpus fine-tunes.

Why production telemetry matters

The 52 production turns mined under ToS §5 (anonymised; salted IP hash; email never linked to training rows) are the first signal the model has seen that wasn't generated by another model. They cover the exact distribution of prompts real learners issue (matrix inverse, DFA, Bayes, vector fields, group theory) rather than what an LLM teacher imagines learners would issue. This is the substrate the model needs to learn to specialise; v5.1 is the start of that loop.

---

Evaluation

20-prompt held-out battery, scored by xml-parse + drawable-element presence:

Full per-prompt results in runs/v5.1_practical_test/summary_rescore.json of the source repo.

Intended use, limitations, license

Same as v4: generating structured math figure specs (SVG + narration + math_claims) from natural-language prompts. English-only training. Adapter only (base downloads separately ~14 GB). MIT licence.

Citation

A peer-reviewed paper describing this work has not yet been published. If you'd like to cite the model in the meantime, please use the model's Hugging Face URL together with the release version in your bibliography:

khayyam-math/khayyam-math-qwen2.5-7b-v5.1
https://huggingface.co/khayyam-math/khayyam-math-qwen2.5-7b-v5.1
Released 2026-05-23, MIT licence.