wyrmling-110M-mtg-dsl

wyrmling-110M — a from-scratch intent→DSL model for Magic: The Gathering mechanics

wyrmling is unofficial Fan Content permitted under the Fan Content Policy. Not approved/endorsed by Wizards. Portions of the materials used are property of Wizards of the Coast. © Wizards of the Coast LLC. Card data provided by Scryfall. Not affiliated with either.

wyrmling is a tiny decoder-only language model built entirely from scratch — its own architecture, its own tokenizer, no inherited weights — for exactly one task: translating a structured card intent into the mtg-compiler DSL, a typed s-expression representation of Magic card mechanics. It speaks no English. The full story is in the blog series Teaching a model to write Magic cards by building the compiler first, especially article 4 (shadowing).

This is the 110M headline model. The size-ladder rung is wyrmling-225M-mtg-dsl — same recipe, same data, and (see below) no better.

Architecture: SwiGLU MLP, RoPE, QK-norm, untied embeddings, logit soft-cap. Optimizer: Muon + AdamW. Tokenizer: custom BPE, ~12k vocabulary, trained only on intent→DSL text so DSL concepts land as single semantically-loaded tokens.

Training

• Pretraining: 18,000 steps (≈0.295B token-positions, ctx 512) on the v8 DSL corpus mixed with 28% grammar-sampled rare-dense data — the optimum from the rare-dense sweep (the anti-shadowing lever; see the blog's article 4).
• SFT: 2 epochs (23,000 steps) on the v8 intent→DSL pairs.
• A full cycle (pretrain + SFT) takes ≈4 h on one AMD R9700.

Evaluation

One held-out v8 test (5,051 cards), free decode, matched generation budgets, Wald 95% CI (docs/finetune/CROSS_TIER_EVAL.md in the repo):

Doubling parameters buys nothing here — the model is data-starved, not capacity-starved. That flat line is the point of the release.

Input / output format

Input is a structured brief (not free prose), output is DSL:

Type: Artifact | Colors: W, R | Cost: {1}{W}{R} | Stats: 1/1
Concept: A creature that gains power and toughness when equipped, and
gains deathtouch if it is a Human and equipped. It costs {2} to equip.

(card_abilities
  (static (pt-mod (selector :type CREATURE :states (list EQUIPPED)) +1 +1))
  (static (gain-ability (it) (DEATHTOUCH))
          :condition (is-type :keyword AS :subtype HUMAN :subject-ref EQUIPPED_CREATURE))
  (keyword EQUIP :cost ((mana "{2}"))))

Validate/render outputs with the MIT-licensed compiler in the repo.

Repository layout

config.json           # WyrmlingHFConfig (vLLM plugin, MTG-E83)
model.safetensors     # verbatim weight dump (embed.weight, blocks.N.attn.qkv.weight, …)
tokenizer.json        # custom BPE, 12,034 entries incl. added tokens
tokenizer_config.json
wyrmling-sft-final.pt # original {config, model} training checkpoint

The .pt loads with the compiler repo's TorchGCDGenerator.from_checkpoint() (grammar-constrained decoding); the safetensors + config load through the repo's vLLM plugin (src/wyrmling/). Neither is a stock transformers architecture — you need the compiler repo either way.

Limitations (read before using)

• Shadowing: nodes seen once in training are missed ~100% of the time; the rare tail is the model's known failure mode and the subject of the research.
• Counterfeit-enum era: the v8 training language still contained 117 counterfeit enum values (retired in !462); scores above are measured on that same language. v9-trained checkpoints will supersede these.
• DSL only — no English, no card names, no flavor text, no art.
• Intended for research on specialization/shadowing and for free fan content. The MIT grant covers this project's rights in the weights; it does not license Wizards of the Coast IP, and commercial use of that IP is not permitted by the Fan Content Policy.

History

Previously distributed as the wyrmling-110M/ subfolder of Tagashy/wyrmling-mtg-dsl. Split into per-variant repos so each model has its files at the repo root.