daedalus-designer-v3

DAEDALUS Designer v2 — Adversarial Auction-Mechanism Design

A 1.5 B-parameter LLM that designs auction mechanisms robust to a population of strategic adversaries (colluders, shaders, dropouts, exploiters). Trained with GRPO (TRL) on the laksh718/Daedalus-Env OpenEnv environment using Unsloth + 4-bit + LoRA.

---

1. What this model does

Given a partial observation of an auction market — recent (welfare, fairness, participation) outcomes, round number, episode length — the model emits a structured JSON mechanism:

{
  "auction_type": "second_price",
  "reserve_price": 0.18,
  "reveal_reserve": false,
  "reveal_competing_bids": false,
  "reveal_winner_identity": true,
  "reveal_clearing_price": true,
  "reveal_bid_distribution": false,
  "shill_penalty": 1.2,
  "withdrawal_penalty": 0.6,
  "collusion_penalty": 1.9,
  "coalition_policy": "penalize_suspected"
}

The mechanism is then used — the env runs 5 market rounds against an adaptive adversarial population, and scores the result on the composite reward R = welfare_ratio · fairness_score · participation_rate · stability_score.

This is the inverse of the usual RL setup: the model is the referee, not the player.

---

2. Model details

---

3. Training details

Pipeline

Qwen2.5-1.5B-Instruct (4-bit, Unsloth)
        |
        | LoRA r=16 on attn + MLP
        v
   +--- SFT on synthetic (prompt, valid_mechanism) pairs
   |    (teaches JSON shape only; ~300 steps)
   v
   GRPO (TRL) on DAEDALUS env, 50 steps
   reward = format_reward + welfare + fairness + composite
        |
        | merge_and_unload + push_to_hub_merged
        v
kabilesh-c/daedalus-designer-v2 (this checkpoint)

Hyperparameters

Training signals (from training_history.json)

Plots and per-step CSV are served from the live Space at /plots/.

---

4. Evaluation

30-episode baseline boxplot on the env's true composite reward (no training-time shaping):

The +0.108 gap (≈ +33 % relative) is the structural signal the model is trained to find. The trained model's online behaviour can be inspected directly in the live Space — every /api/design call hits this checkpoint.

A typical stage-3 (mixed shaders + colluders) output:

{
  "auction_type": "second_price",
  "reserve_price": 0.18,
  "reveal_competing_bids": false,
  "reveal_clearing_price": true,
  "reveal_winner_identity": true,
  "collusion_penalty": 1.9,
  "shill_penalty": 1.2,
  "withdrawal_penalty": 0.6,
  "coalition_policy": "penalize_suspected"
}

Three things to read off this:

1. Picks second-price (truthful in static, robust to non-VCG-aware bidders).
2. Hides bid distribution but reveals clearing price — gives honest bidders calibration signal while starving cartels of enforcement signal.
3. Penalty ranking: collusion (1.9) > shill (1.2) > withdrawal (0.6) matches the relative cost of each pathology in this population.

---

5. Inference

5.1 Hosted (no setup) — call the live Space

import requests

BASE = "https://kabilesh-c-daedalus-env.hf.space"

# Reset and read the initial observation
r = requests.post(BASE + "/reset",
                  json={"session_id": "demo", "n_agents": 8, "episode_length": 10})
obs = r.json()["observation"]

# Ask the trained designer for a mechanism
r = requests.post(BASE + "/api/design", json={
    "round_number": obs["round_number"],
    "episode_length": obs["episode_length"],
    "market_outcomes": obs.get("market_outcomes", []),
})
print(r.json()["mechanism"])

# Apply it and observe the reward
r = requests.post(BASE + "/step",
                  json={"session_id": "demo", "action": r.json()["mechanism"]})
print("R =", r.json()["reward"])

5.2 Local — transformers + huggingface_hub

Important: this repo was originally pushed via Unsloth's save_pretrained_merged(), which leaves a leftover adapter_config.json whose base_model_name_or_path points at ./sft-merged (a path that only exists on the training filesystem). Plain from_pretrained(repo_id) will follow that pointer and crash with HFValidationError: Repo id must use alphanumeric chars. Use snapshot_download with ignore_patterns to skip the adapter file:

from huggingface_hub import snapshot_download
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

local_dir = snapshot_download(
    repo_id="kabilesh-c/daedalus-designer-v2",
    ignore_patterns=["adapter_config.json", "adapter_model.safetensors"],
)

tok = AutoTokenizer.from_pretrained(local_dir)
model = AutoModelForCausalLM.from_pretrained(
    local_dir,
    torch_dtype=torch.bfloat16,   # fp16 if no bf16
    device_map="auto",
)

system = (
    "You are an auction mechanism designer. Given a market observation, "
    "output ONLY a JSON object matching the DaedalusAction schema."
)
user = (
    "round_number: 3 / 10\n"
    "recent_outcomes: [{welfare:0.7, fairness:0.4, participation:1.0}]\n"
    "Respond with the JSON only."
)
prompt = tok.apply_chat_template(
    [{"role": "system", "content": system},
     {"role": "user",   "content": user}],
    tokenize=False, add_generation_prompt=True,
)
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=180, do_sample=False, temperature=0.0)
print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True))

5.3 Local — full env rollouts via inference.py

The repo at kabilesh-c/Daedalus-Env ships an inference.py that reproduces the §4 baseline numbers:

# one-shot mechanism for a fresh env
python inference.py

# 30 episodes trained-vs-random (writes inference_results.json)
python inference.py --n-episodes 30 --baseline

# point at a different checkpoint
python inference.py --repo-id kabilesh-c/daedalus-designer-v2

---

6. Intended use & limitations

Intended use: demonstration / research on LLM-driven mechanism design. The model is trained to output well-formed DaedalusAction JSON for the DAEDALUS env; it has no production guarantees.

Limitations:

• Trained for only 50 GRPO steps — the +50 step lift over a randomly- initialised LoRA is real but small in absolute terms (composite-reward mean from −0.594 → −0.579).
• The "format reward" dominates early steps; malformed JSON is heavily penalised, so the model is schema-locked but not strategically perfect. On stage-4 (full-adversarial) populations it occasionally emits coalition_policy: "allow", which craters the run.
• Reward is engineered for the DAEDALUS env's specific multiplicative composite. The model is not guaranteed to produce sensible mechanisms outside this rubric.
• Inherits any biases / failure modes from Qwen/Qwen2.5-1.5B-Instruct.

---

7. Citation

@misc{kabilesh2026daedalus,
  title  = {DAEDALUS: Training an LLM to Design Auction Markets via Adversarial RL},
  author = {Laksh Krish Kabilesh},
  year   = {2026},
  url    = {https://huggingface.co/spaces/kabilesh-c/Daedalus-Env},
}

---

Made by Laksh Krish Kabilesh.