Qwen2.5-Coder-32B-Palace-LoRA

Qwen2.5-Coder-32B-Palace-LoRA

Paper: Beyond Schema Conformance with Executable Verification for Scientific Configuration Repair (NeurIPS 2026 Workshop on AI for Verifiable Coding)

A QLoRA adapter that specializes Qwen/Qwen2.5-Coder-32B-Instruct for authoring and repairing Palace configuration files. Palace is an open-source finite-element solver for computational electromagnetics, and its runs are driven by a strict JSON configuration. General code models handle this format poorly because it is niche and its constraints are unforgiving — a wrong field name, type, or unit silently changes the physics. This adapter teaches the base model Palace's exact schema vocabulary and idioms so it produces configurations that validate against the schema and are accepted by the solver.

The adapter is trained against Palace v0.14.0 artifacts.

What it does

• Authoring — write a Palace JSON config from a natural-language description of a problem (electrostatic, magnetostatic, or driven electromagnetics).
• Repair — given a broken config, fix schema violations, wrong types, invalid enums, dangling mesh references, and missing required sections.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "Qwen/Qwen2.5-Coder-32B-Instruct"
ADAPTER = "rwmasood/Qwen2.5-Coder-32B-Palace-LoRA"

tok = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)

messages = [
    {"role": "system", "content": "You write and repair Palace configuration files."},
    {"role": "user", "content": "Write a Palace electrostatic configuration that computes the "
                                "capacitance of a parallel-plate capacitor. Mesh: mesh/plate.msh, "
                                "one terminal on attribute 3, ground on attribute 4, vacuum domain "
                                "on attribute 1."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=1024, do_sample=False)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Serving with vLLM

vllm serve Qwen/Qwen2.5-Coder-32B-Instruct \
  --enable-lora \
  --lora-modules palace=rwmasood/Qwen2.5-Coder-32B-Palace-LoRA \
  --max-lora-rank 64

Then request the model name palace.

Training

• Base model: Qwen/Qwen2.5-Coder-32B-Instruct
• Method: QLoRA (4-bit NF4 base), LoRA rank 64, α=128, dropout 0.05, applied to all linear projections (q,k,v,o,gate,up,down)
• Data: 3,296 instruction/response pairs (199 held out for validation) derived from Palace's JSON schema, documentation, and shipped example configs, rendered into config-authoring and targeted-repair tasks. ChatML template, loss on responses only.
• Schedule: 3 epochs, sequence length 8192, effective batch size 16 (micro-batch 4 × grad-accum 4), LR 1e-4 cosine, warmup 0.03, paged AdamW-8bit, bf16, seed 17.
• Frameworks: PEFT 0.19.1, Axolotl 0.17.0, Transformers.

Intended use and limitations

Intended for engineers and researchers generating or fixing Palace configuration files. Outputs are JSON and are meant to be run through the solver, not consumed as prose.

Always verify generated configs before trusting a result. Validate against the Palace JSON schema and run palace --dry-run, which catches structural and parser-level problems. Note that a config can pass both of these and still be physically wrong — for example a mis-scaled length unit (L0) produces a config that is schema-valid and runs to completion yet computes a very different physical system. Where a reference or an analytic answer exists, check the numerical output against it. The adapter targets Palace v0.14.0; other versions may differ in schema.

Citation

@inproceedings{masood2026beyond,
  title     = {Beyond Schema Conformance with Executable Verification for Scientific Configuration Repair},
  author    = {Masood, Wasif and Noor, Shoaib},
  booktitle = {NeurIPS 2026 Workshop on AI for Verifiable Coding (VeriCodeGen)},
  year      = {2026}
}

License

Released under the Apache 2.0 license, matching the base model. You must also comply with the license of Qwen/Qwen2.5-Coder-32B-Instruct.