polymarket-settlement-risk-v5

Polymarket Settlement-Risk Predictor — v5 (LoRA)

A LoRA adapter (rank 16) for Qwen/Qwen3.5-4B that predicts whether a Polymarket market will attract at least one UMA settlement dispute, using only information available when the market was created plus retrieved historical analogues.

Trained with reinforcement learning (Prime-RL) on a class-balanced, era-extended dataset with a proper-scoring (Brier) reward. Fifth and best iteration of the series.

Results (held-out, single run)

• On familiar markets: ties a frontier reasoning model on ranking (AUC ~0.97) while making far better thresholded decisions, at ~1/230th the inference cost.
• On fresh markets (a month past training): AUC ~0.70 — and beats the frontier model there (0.70 vs 0.66). Dispute risk on new markets is only weakly predictable from text at any model scale; retrain monthly.

Usage

Serve with vLLM (enable_lora=True, max_lora_rank=16); prompts must include the historical-analogue block (see the app repo). Sampling: temp 0.2 / top_p 0.8 / top_k 20 / presence_penalty 1.5 / max 384 tokens. Calibrate raw output with Platt a=1.8404, b=-1.9985.

Honest limit: a screening signal, not a guarantee. Predicts UMA disputes specifically.