sqrl-4b

sqrl-4b

sqrl-4b is a 4B-parameter agentic text-to-SQL model built on Qwen/Qwen3.5-4B. It answers natural-language questions over SQLite databases by optionally probing the database first — running read-only exploration queries to check value formats, join paths, and filters — before committing to a final SQL answer.

It was trained in two stages:

1. SFT distillation — fine-tuned on ~10.2k execution-verified agentic trajectories sampled from a much stronger RL-trained 35B teacher (Qwen3.6-35B-A3B, 70.3% BIRD-dev EX) on cleaned BIRD-train + Spider-train questions. Only trajectories whose final SQL execution-matched the gold answer were kept.
2. CISPO RL — online reinforcement learning (group-relative advantage, group size 8) with execution-match reward on the same cleaned BIRD+Spider question pool, with async dynamic sampling. Best checkpoint at step 80.

Results

All numbers measured with the agentic harness at temperature 0.7. BIRD-dev is the full 1534-question set; Spider-test is 2147 questions. pass@1 is the mean single-sample exec-accuracy; vote@k is execution-clustered majority voting over k samples (group candidates by identical execution result set, return the largest cluster's query).

Family comparison on identical seeds (BIRD-dev pass@1 / vote@8): sqrl-4b 64.6 / 68.8 · sqrl-9b 66.6 / 69.8 · sqrl-35b-a3b 68.7 / 70.6.

Notes from the measurement campaign:

• Voting buys the 4B +4.3 points over single-pass on BIRD (the largest voting gain in the family — smaller models keep more sample diversity). vote-of-4 captures ~85% of that gain at half the cost.
• Voting saturates by k≈8–16. Past that, more samples add nothing: BIRD peaks at vote@16 (69.4) and Spider is flat from vote@8 onward (~86%). Extra test-time compute is better spent on a smarter selector than on more votes.
• Base Qwen3.5-4B scores ~52% on the same harness; the pre-RL SFT checkpoint ~63.6% on BIRD (80.3% on Spider-test — RL lifts Spider single-shot to 84.0%).
• Vote unanimity is a strong free confidence signal: when all 8 samples agree the answer is right ~87% of the time; narrow/tied votes are right ~19-43% — useful for routing or escalation.

How it works — the agentic protocol

The model is trained to emit exactly one action block per turn, after a brief reasoning summary:

• <sql> SELECT ... </sql> — a read-only exploration query. Execute it against the database and feed the result back as a user turn wrapped in <observation> tags. The model continues (up to a step budget).
• <answer> SELECT ... </answer> — the final SQL. Execute and return.

Many questions are answered directly (zero exploration turns); the model learned to explore only when seeing real data would change its answer (value-format checks, ambiguous joins).

Usage

Serve with vLLM

vllm serve feyninc/sqrl-4b --served-model-name sqrl-4b \
  --data-parallel-size 4 --gpu-memory-utilization 0.90 --max-model-len 32768

Important: do not enable a reasoning parser (e.g. --reasoning-parser qwen3). The model's post-</think> content carries the action protocol; stripping/rerouting the think block breaks it (measured ~12-point EX drop). Parse message.content — if it contains a </think>, take everything after it.

Recommended sampling: temperature 0.7, top_p 0.95 (matches training); greedy also works.

Prompt format

System prompt (fill {schema}, {evidence}, {max_steps}):

<role>
You are an expert data analyst, fluent in SQL, with a meticulous eye for
matching a question's intent to the exact tables, columns, and stored value formats of
a database.
</role>

<task>
Translate the user's natural-language question into a SQL query that answers
it, using the database schema in <schema> and any domain hints in <evidence>. You can
run read-only queries against the database to inspect it before giving your final
answer.
</task>

<database_engine>
SQLite
</database_engine>

<schema>
{schema}
</schema>

<evidence>
{evidence}
</evidence>

<protocol>
Think through the problem internally first. Then, in your response, write a
BRIEF summary of your reasoning — 2-4 sentences stating which tables and columns are
relevant, the joins and filters, and any exact value-format detail. Be decisive: state
the plan once, do not second-guess or restate. End your message with EXACTLY ONE action
block (and nothing after it):

  <sql> a read-only query to inspect the database </sql>
     Use this when uncertain and you want to see real data before answering — e.g.
     confirm a value's exact stored format, check a filter actually matches rows,
     sanity-check an intermediate result, or verify a join. You will see the result
     rows and then continue.

  <answer> your final SQL query </answer>
     Use this once you are confident. It is executed and scored, and the task ends.

Examples:
  I'll check the exact county value before filtering, since the format may vary.
  <sql> SELECT DISTINCT `County Name` FROM frpm LIMIT 10 </sql>

  The schools are in the frpm table; I'll select `School Name` and filter `County Name`
  to 'Alameda'. The schema is clear, so I can answer directly.
  <answer> SELECT `School Name` FROM frpm WHERE `County Name` = 'Alameda' </answer>
</protocol>

<rules>
- Use only the tables and columns defined in <schema>.
- Quote identifiers containing spaces or special characters with backticks.
- Return exactly the columns the question asks for — no more, no fewer.
- Use the hints in <evidence> to resolve ambiguous terms and value encodings.
- If you already know the correct query, go straight to <answer> — investigating is
  optional; only run <sql> when seeing the data would actually change your answer.
- The action block must be the LAST thing in your message; do not discuss the tags
  themselves in your reasoning.
- You have at most {max_steps} <sql> steps; after that you must give <answer>.
</rules>

First user turn:

Question: {question}

Reason about it, then give <sql> to investigate or <answer> to finish.

After executing an exploration <sql>, feed the result back as a user message:

<observation>
{tab-separated result rows, or the error message}
</observation>
Continue with <sql> or <answer>.

{schema} is a readable dump of the SQLite schema (CREATE-TABLE-like listing of tables/columns/types). {evidence} is the BIRD external-knowledge hint, or (none provided). max_steps=5 was used in training. When the step budget is exhausted, send You must finish now. Give your <answer>. as a user turn.

Minimal driver loop

import re
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

messages = [{"role": "system", "content": system_prompt},
            {"role": "user", "content": user0}]
for step in range(6):
    r = client.chat.completions.create(model="sqrl-4b", messages=messages,
                                       temperature=0.7, top_p=0.95, max_tokens=8192)
    content = r.choices[0].message.content
    if "</think>" in content:                       # strip inline think scratchpad
        content = content.split("</think>")[-1].strip()
    messages.append({"role": "assistant", "content": content})
    ans = re.findall(r"<answer>(.*?)</answer>", content, re.S)
    if ans:
        final_sql = ans[-1].strip(); break
    sql = re.findall(r"<sql>(.*?)</sql>", content, re.S)
    if not sql:
        break
    obs = run_readonly(sql[-1].strip())             # your sqlite executor
    messages.append({"role": "user",
                     "content": f"<observation>\n{obs}\n</observation>\n"
                                "Continue with <sql> or <answer>."})

Checkpoint notes

• This repo contains the full merged HF checkpoint (LoRA rank 64 merged into the base). Qwen3.5-4B ships with tied embeddings; training touched only the output head, so this checkpoint unties them (tie_word_embeddings=false): embed_tokens is the pristine base matrix and lm_head carries the trained head. Loads with standard transformers / vLLM — no custom code beyond trust_remote_code if your transformers version requires it for Qwen3.5.
• The vision tower is inherited from the base model unchanged. The model is usable as a standard VL checkpoint, but SQL training was text-only.
• Text-to-SQL training data: BIRD-train and Spider-train, execution-filtered and semantically cleaned (3-model LLM-judge panel). BIRD-dev and Spider-test were never trained on.

Intended use & limitations

Built for SQLite text-to-SQL with schema + optional evidence in context. Works best with the exact prompt protocol above (it was both SFT'd and RL'd under it). Not tuned for other SQL dialects; identifier quoting follows SQLite backtick conventions. As with all text-to-SQL models, execute generated SQL read-only and validate before acting on results.