aurelius-v2

Aurelius v2 — verifier-native serving system (model card)

Base: Qwen/Qwen3-Coder-30B-A3B-Instruct (Apache-2.0, MoE 30.5B total / 3.3B active). Clean-teacher provenance ✓. What v2 is: the strong open base + a thin, MEASURED serving layer — one lever per axis, each the one that actually worked.

Results (measured on this base)

Capability lever = CAPACITY (the real win)

30B-A3B vs 14B, zero training: MATH-500 +33pp, GSM8K +20pp, MATH-L5 greedy 65.7 vs ~47. The base swap is the capability gain; the serving layer cashes the cheap selection/repair headroom on top.

What worked / what didn't (honest)

• ✅ Capacity (bigger base) — the capability lever.
• ✅ maj@N self-consistency (math) — free, +8.2pp on hard math. SHIPPED.
• ✅ best-of-N + repair (code, exec-verifier) — +10.5pp MBPP on v1; carried to v2.
• ✗ LLM-as-verifier rerank (math) — NULL: verifier@8 73.1% ≈ maj@8 73.9% (1 worse). Zero-shot self-verification does not beat majority voting; the remaining ~7.4pp selection gap (maj→oracle 81.3%) needs a TRAINED verifier/RLVR.
• ✗ SFT-for-reasoning — confirmed neutral (v1: MATH 46.5%=base 46.5%). Reasoning is inherited, not SFT-able.
• ✗ RLVR on frozen 8B / flywheel-into-greedy — prior nulls; not revisited.

Usage

solve("math", "<problem>")                      # -> {answer, confidence, votes}
solve("code", "<task asking for one ```python block>", tests="<assert lines>")   # -> {code, how, passed}

maj@N: no verifier needed (ships anywhere). Code best-of-N: needs caller-supplied tests as the verifier. Compute ~Nx greedy. Math max_new=4096 (think=1 needs room; check no-box). Apache → public-release-eligible (keep private until gated). Future work (gated on compute): trained verifier / RLVR-for-selection for the residual ~7.4pp.

Limitations

• Code lever is only as strong as the caller's tests. Tests stricter than the true spec (e.g. demanding a list when a correct algorithm returns a tuple) report failure even on correct code — spec the output type in the prompt and/or normalize in the tests. solve_code surfaces feedback on failure to debug this.
• maj@N self-consistency helps only where the model samples the right answer sometimes (no help on hard-capability problems it never gets — ~15% of MATH-L5 never-recovered).

Files

• aurelius_serve.py — self-contained serving (maj@N math + best-of-N+repair code)
• *.gguf — quantized weights for local serving (if present)

from aurelius_serve import AureliusServe
s = AureliusServe()
print(s.solve('math', 'What is 12*13?'))