quantumgpt-354m

QuantumGPT-354M: Quantum Circuit Generation Model

QuantumGPT-354M is a GPT-style language model (354.1M parameters) trained from scratch on quantum circuit description → OpenQASM 2.0 pairs. It is the third model in the QuantumGPT scaling series, scaling model depth and width while holding training data constant at 21,208 samples.

Key finding: QuantumGPT-124M-v2 outperforms this model on all primary metrics. At this data scale (1.75M tokens), the binding constraint is data coverage, not model capacity. See the scaling series table below.

---

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("merileijona/quantumgpt-354m")
tokenizer = AutoTokenizer.from_pretrained("merileijona/quantumgpt-354m")

prompt = "<|user|>Create a Bell state with two qubits<|end|>\n<|assistant|>"
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=300,
    do_sample=True,
    temperature=0.8,
    top_k=50,
    repetition_penalty=1.1,
    pad_token_id=tokenizer.eos_token_id,
)

text = tokenizer.decode(outputs[0], skip_special_tokens=False)
response = text[len(prompt):]
if "<|end|>" in response:
    response = response[:response.index("<|end|>")]
print(response.strip())

---

Model Details

Architecture

Training Configuration

Overfitting

Severe overfitting begins around step 1,400 (train/val gap > 0.15) and reaches +0.34 by the final step. The best checkpoint at step 1,100 was used for conversion. The data-to-parameter ratio (~4.9 tokens per parameter) is well below the Chinchilla-optimal ratio of ~20, making this model data-constrained.

---

Benchmark Results

Evaluated on QuantumGPT Benchmark v1.0 — 100 prompts, 50 ID / 50 OOD, 3 difficulty tiers, k=5 samples, seed=42. Prompt suite hash: ee2da8a57e683af2464eb7a4eada0898.

Scaling Series Comparison

Conclusion: Increasing parameters 2.9× while holding data constant yields no improvement over QuantumGPT-124M-v2. Data scaling outperforms model scaling at this regime.

Failure Mode Breakdown (500 samples)

WRONG_QUBITS (circuits with incorrect qubit counts) is the dominant failure mode and is unaffected by model scale.

---

Prompt Format

<|user|>{natural language description}<|end|>
<|assistant|>{OpenQASM 2.0 circuit}<|end|>

Delimiters are literal text tokens, not special tokenizer tokens.

---

Limitations

1. Data-constrained overfitting — model is severely undertrained relative to its parameter count; generalisation is limited to what the best checkpoint captures at step 1,100.
2. WRONG_QUBITS — ~31% of outputs have incorrect qubit counts regardless of prompt specification.
3. Semantic correctness — 59pp gap between syntax and semantic validity at pass@5; not improved over smaller models.
4. Synthetic training data — all training circuits generated by LLM (xAI Grok), not from real quantum programs.
5. No hardware validation — requires transpilation and validation before execution on real quantum hardware.

---

Intended Use

✅ Research baseline for quantum circuit generation scaling studies
✅ Comparison point for data-vs-parameter scaling analysis
✅ Educational demonstrations of QASM generation

❌ Production quantum computing workflows
❌ Use cases where QuantumGPT-124M-v2 is available (it performs better)

---

Citation

@misc{quantumgpt354m,
  author    = {Merilehto, Juhani},
  title     = {QuantumGPT-354M: Parameter Scaling Study for Quantum Circuit Generation},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/merileijona/quantumgpt-354m},
  note      = {354.1M parameter GPT trained on quantum-circuits-21k.
               Data scaling outperforms model scaling at this regime.}
}

---

Model Card Authors

Juhani Merilehto

• HuggingFace: @merileijona
• GitHub: @juhanimerilehto
• Affiliation: University of Vaasa, School of Management; University of Turku, Faculty of Technology

License

MIT License

Acknowledgments

• Training framework: Andrej Karpathy's nanoGPT / nanochat architecture
• Data generation: xAI Grok API
• Tokenizer: Standard GPT-2 BPE (HuggingFace GPT2TokenizerFast)
• Validation: Qiskit OpenQASM 2.0 parser
• Hardware: NVIDIA RTX 4070 12GB / AMD Ryzen 9 5950X / 128GB RAM

Additional Resources

• Training dataset: merileijona/quantum-circuits-21k
• Better model: merileijona/quantumgpt-124m-v2

---

Model Version: 1.0 Release Date: March 2026