quantumgpt-124m-v3

QuantumGPT-124M-v3 (ctx512)

Third-generation QuantumGPT: a GPT-2 small (124M) model fine-tuned to generate OpenQASM 2.0 circuits from natural-language descriptions, with the context window extended from 256 to 512 tokens via position-embedding extension.

QSS-Core v0.1: 97.70 / 97.28 (two-seed replication) — up from 91.61 (v2) and 75.00 (v1 baseline) on the same 120-sample stratified development benchmark.

Model details

Benchmark results (QSemBench-dev-v0.1, n=120 stratified, greedy decode)

Scoring: qiskit qasm2 parse/compile + Aer statevector fidelity ≥ 0.99 vs reference circuit, entanglement check where reference entangles, prompt-perturbation robustness probes. Full methodology in the technical report below.

*v1 baseline reported under pre-cap-lift conditions; see report for details.

Capabilities (qualitative evaluation)

Strong on: Bell/GHZ-style entanglement, teleportation structure, Deutsch-Jozsa architecture, three-qubit error-correction encoding, mid-circuit measurement with classical control (if(c==1)), parameterized rotations, novel state preparations adjacent to training distribution (e.g., plausible W-state synthesis with correct arccos(1/√3) angles).

Weak on: negation constraints ("WITHOUT any entangling gates" ignored), compound multi-part instructions (partial completion), physics-level self-checking (no-cloning request produces a circuit instead of refusal), casual/typo-heavy phrasing degrades semantic accuracy.

Limitations

• Development benchmark shares its generator with the training corpus; truly held-out evaluation requires newly generated circuits.
• Trained exclusively on synthetic Grok-generated circuits validated by qiskit parse checks.
• No alignment/refusal behavior: any prompt returns some QASM program, meaningful or not.
• 512-token context is still limiting for large-scale algorithms (Shor-9 encoding sits at the edge).
• English prompts only.

Intended use

Research on small-scale code-generation models for quantum computing, education, and as a base for further scaling experiments. Not intended for production quantum-circuit synthesis without human verification of every generated circuit.

Reproducibility

Training and evaluation harness: deterministic orchestrator with sealed experiment packets. Key artifacts: champion lineage QEXP-100006 → QEXP-200042/200043; replication spread between seeds: 0.41 QSS points. See technical report in this repository for full experiment history.