qwen3.5-2b-grpo-synsql

<h1 align="center">M_SynSQL: a 2B text-to-SQL agent trained on SynSQL-2.5M</h1> <p align="center"> <a href="https://trl-lab.github.io/sqale"><b>Project page</b></a> · <a href="https://huggingface.co/trl-lab/qwen3.5-2b-grpo-sqale"><b>M_SQaLe</b></a> · <a href="https://huggingface.co/collections/trl-lab/sqale-project"><b>All three models</b></a> · <a href="#citation"><b>Citation</b></a> </p>

M_SynSQL is one of the three text-to-SQL models from the paper SQaLe: a large realistic dataset to empower small specialised text-to-SQL models. It is Qwen3.5-2B trained with GRPO on SynSQL-2.5M, using the same recipe as M_SQaLe, the paper's model trained on SQaLe. SynSQL-2.5M holds 2.5 million samples, more than any other text-to-SQL corpus the paper compares, and the paper uses this model to measure what the same training achieves on it. Like its siblings, M_SynSQL works as an agent. It explores the database through tools, tests its query and then submits it, and the schema is never part of its prompt.

Quickstart

The model is an agent, so it needs a loop that executes its tool calls against a database and feeds the results back. sqale_agent.py in this repository is a self-contained reference loop that reproduces the training environment: the same prompt, tools, turn budget and final-answer fallback. It opens your database read-only.

pip install vllm huggingface_hub
hf download trl-lab/qwen3.5-2b-grpo-synsql sqale_agent.py --local-dir .

python sqale_agent.py --model trl-lab/qwen3.5-2b-grpo-synsql \
  --db path/to/database.sqlite \
  --question "Which supplier shipped the most back-ordered items last quarter?" \
  --trace

From Python, one agent can answer many questions with a single loaded engine:

from sqale_agent import SQaLeAgent, vllm_generate

agent = SQaLeAgent(*vllm_generate("trl-lab/qwen3.5-2b-grpo-synsql"))
episode = agent.run("How many customers placed more than three orders in 2024?", "shop.sqlite")
print(episode["sql"])       # the submitted query
print(episode["ended_by"])  # submit_sql, final_turn, fallback_to_last_working_query, ...

To build your own loop, follow the protocol in The agent loop below.

Highlights

• 44.3% on BIRD dev, +25.0 points over the untrained base model (19.3%), without any BIRD data in training.
• 50.7% on the SQaLe test set (+12.0 points over the base model) and 13.3% on EHRSQL (+5.1 points).
• The most active explorer of the three, at 29.9 tool calls per episode against 22.2 for M_SQaLe.
• Reinforcement learning only. It is trained from the base checkpoint with no supervised warm start, no reasoning traces distilled from a larger teacher, and no test-time scaffolding at evaluation.

<p align="center"> <img src="figures/size_vs_accuracy.png" width="760" alt="Accuracy against model size on 100 moderate SQaLe test questions"> </p> <p align="center">_{Accuracy on 100 moderate questions from the SQaLe test set against model size, with the average compute per question as colour. M_SQaLe, M_BIRD and M_SynSQL are Qwen3.5-2B trained with the same recipe on SQaLe, BIRD train and SynSQL-2.5M; the red arrow is the gain of M_SQaLe over the untrained Qwen3.5-2B. Figure from the paper's appendix.}</p>

The SQaLe model family

The paper trains Qwen3.5-2B three times with one recipe, once per training corpus, to measure how much of a small model's ability comes from the data it is trained on. Base model, agentic environment, tool set, reward, GRPO objective, step count, batch size and evaluation protocol are identical across the three. Only the training corpus and the strength of the length curriculum differ, so the gaps between the models come from the data.

Evaluation

Execution accuracy (%) on 300 questions per benchmark, as reported in the paper. Every model runs in the agentic environment used for training, with the schema withheld and no test-time scaffolding. The SQaLe test split shares no schema with SQaLe's training split and is evaluated over the full schema. EHRSQL is a single-domain benchmark over electronic health records.

Compared with M_SQaLe

SynSQL-2.5M holds more samples than SQaLe, but its schemas are small and sparsely populated, with a median of two rows per table against 69 in SQaLe. On the SQaLe test set, M_SQaLe leads M_SynSQL by 15.6 points at full schema size, and the paper's analyses trace much of the gap to the size and content of the training databases.

Searching large schemas

The SQaLe test questions are evaluated at three database sizes: the gold tables only, the gold tables plus 32 distractor tables, and the full schema. Brackets give the share of episodes in which the model opened every gold table.

SynSQL schemas hold 5–10 tables. With only the gold tables present, M_SynSQL finds all of them in 98.3% of episodes. At full SQaLe schema size it opens every gold table in 83.3% of episodes, against 89.7% for M_SQaLe.

Looking up values

<p align="center"> <img src="figures/value_grounding.png" width="680" alt="How each model obtained the string literals of the gold answer"> </p>

Many questions depend on literals that exist only in the table values. M_SynSQL makes the most tool calls of the three models and looks up 55% of the string literals the gold answer depends on, which is more than M_BIRD (40%) and the base model (35%) and less than M_SQaLe (88%). With two rows per table, its training databases rarely made it necessary to search a large, populated table for a value. When the question does not state the literal, M_SynSQL searches the data for 37% of them and uses the correct value in 39% of cases, against 61% and 55% for M_SQaLe.

The agent loop

What an episode looks like

Question   Which supplier shipped the most back-ordered items last quarter?
Turn 1     {"tool": "list_tables"}
           → suppliers, shipments, order_items, …
Turn 2     {"tool": "join_path", "args": {"table_a": "suppliers", "table_b": "order_items"}}
           → suppliers → shipments → order_items
   ⋮
Turn 7     {"tool": "submit_sql", "args": {"sql": "SELECT s.name …"}}

Each turn opens with a <think> block in which the model plans its next call, and a turn may hold several calls.

Using your own harness

The following details from training matter for accuracy if you build your own loop.

• Use the training prompt verbatim. SYSTEM_PROMPT and USER_TEMPLATE in sqale_agent.py are the exact strings used in training. The system prompt holds the task, the tool list and an execution-plan scaffold. The user turn states that the schema is not shown and gives the question.
• Tool calls are bare JSON. After its <think> block the model writes one or more {"tool": "<name>", "args": {...}} objects with no prose and no code fences. Keep thinking enabled.
• Return results as the next user turn, formatted as Tool result: followed by the result and the remaining budget, for example [4 tool turns left before you must answer].
• Keep earlier reasoning in context. During training every earlier <think> block stayed in the context. The Qwen chat template drops those blocks when a conversation is rebuilt from a message list, so append each completion to the prompt string instead, as sqale_agent.py does.
• Six tool turns, then a final turn. After the sixth tool turn, or after four failed run_query calls in a row, send the exploration-limit message and take the next answer as final. If that answer is empty or does not execute, use the last query that ran without error.
• Sampling. Use temperature 0.6, top-p 0.95 and top-k 20, which are the defaults in generation_config.json, and stop on <|im_end|> and <|endoftext|>. An episode is budgeted at 12,288 tokens, so a context of 16,384 tokens leaves headroom.

Training

Environment. Each training question becomes a tool-use episode over a live SQLite database, and the schema is never placed in the prompt. A graph-aware sampler decides which tables the episode's database holds. It draws a target size between 4 and 64 tables, starts from the gold tables and grows along foreign keys until the target is reached, then shuffles the table order. SynSQL databases are small, so the sampler often returns the full schema. Early in training, a submit gate refuses submit_sql until the same query has been run with run_query. The gate's probability decays from 1.0 to 0 over the first 80 steps, and it is never active at evaluation.

Reward. A query whose result set matches the gold result scores 3.0. A query that executes with a wrong result scores 1.0 + 0.9·F₁ + b, where F₁ is the multiset F₁ against the gold result and b adds 0.1 times the share of joins along declared foreign keys and 0.1 for agreeing with the most common result in the group. A query that parses but does not execute scores 0.5, and anything else scores 0. The executable tier tops out at 2.1, so no partial credit ranks a wrong query above a correct one. Correctness is decided by comparing result sets, with no LLM judge.

Length curriculum. SynSQL's gold queries span a narrower range of lengths than SQaLe's, so M_SynSQL gets a milder curriculum than M_SQaLe, with three buckets instead of four. The bucket edges sit at 149 and 278 characters of masked gold SQL, and the long bucket covers every query from 278 characters up. The training pool is filled with shares of 0.20, 0.25 and 0.55. The first 400 steps draw from the short bucket only. The mixture then moves linearly from (0.40, 0.30, 0.30) to (0.15, 0.20, 0.65) over 1,000 steps and holds there for the remaining 400. This is the only training setting besides the corpus in which it differs from M_SQaLe.

Optimisation. GRPO from the Qwen3.5-2B base checkpoint, with 18 questions × 8 candidates = 144 episodes per update for 1,800 steps. This repository holds the last checkpoint. BIRD dev was monitored during training and not used to select the checkpoint.

<details> <summary><b>All hyperparameters</b> (identical for the three models except the curriculum)</summary> </details>

Intended use and limitations

• M_SynSQL answers natural-language questions over SQLite databases through tool access. It is a research model released with the paper, mainly as the SynSQL-trained reference for M_SQaLe.
• On databases with many tables it finds the relevant tables and values less reliably than M_SQaLe, which is the better choice of the three for large schemas.
• It makes more tool calls per episode than its siblings, so expect longer episodes.
• It expects the agent loop described above. Putting the DDL in a single prompt and asking for SQL is a different setting from the one it was trained in.
• It was trained and evaluated on SQLite only.
• At 2B parameters it still trails much larger models. The untrained Qwen3.6-27B scores 76.0 / 69.3 / 55.0 on the three benchmarks above, and EHRSQL's clinical questions remain hard for every 2B model in the paper.
• The model's SQL runs against your data, so give it read-only access, as sqale_agent.py does.
• The checkpoint is a full Qwen3_5ForConditionalGeneration export and carries Qwen3.5-2B's vision tower unchanged. Only the language model was trained, and only text was evaluated.

Citation

If you use this model, please cite the SQaLe paper:

@misc{wolff2026sqale,
  title  = {{SQaLe}: A Large Realistic Dataset to Empower Small Specialised Text-to-{SQL} Models},
  author = {Wolff, Cornelius and Gomm, Daniel and Hulsebos, Madelon},
  year   = {2026}
}

Authors: Cornelius Wolff and Daniel Gomm (University of Amsterdam, Centrum Wiskunde & Informatica), Madelon Hulsebos (Centrum Wiskunde & Informatica). Questions and feedback are welcome in the Community tab of this repository.

The model builds on Qwen3.5-2B and is trained on SynSQL-2.5M from the OmniSQL project. The evaluation also uses BIRD, EHRSQL and the SQaLe test set, whose schemas come from SchemaPile. We thank their authors for making them available.