cl-ds

Common Lisp Macro Transformations

A fine-tuning dataset for training models to generate Common Lisp macros. Each example is a (before-code) → (macro-definition) → (after-expansion) triple.

Idea

Instead of fine-tuning a model to "write code", fine-tune it to generate CL macros — code that writes code. The model learns to recognize AST patterns and generate transformations, not final output.

Sources

• Let Over Lambda — Doug Hoyte's production macro collection (thephoeron/let-over-lambda)
• On Lisp — Paul Graham's classic Common Lisp macro utilities

Dataset Structure

Each record contains:

• instruction — Task description with the code pattern to address
• input — The "before" code showing the pattern that needs a macro
• output — The defmacro form that solves it
• category — Macro category (capture-management, anaphoric, dispatch, control-flow, DSL, compiler-macro, efficiency, scope)
• technique — Comma-separated techniques used (gensym, nested-backquote, dlambda, anaphor, code-walking, symbol-macrolet, defsetf, tagbody-go, once-only, macrolet, compiler-macro, recursive-expansion)
• complexity — basic, intermediate, or advanced
• quality_score — Classifier score from 0.0 to 1.0

Categories

Use for Fine-tuning

The data is in instruction-input-output JSONL format, ready for fine-tuning:

from datasets import load_dataset
ds = load_dataset("j14i/cl-macros", split="train")

Target model size: ≤ 30B parameters (the domain is narrow — pattern matching on ASTs and transformations — so a smaller model suffices).