Qwen3.5-0.8B-OptiQ-4bit-text-to-sql

Qwen3.5-0.8B Text-to-SQL (MLX LoRA)

A Qwen3.5-0.8B model (the smallest member of the Qwen3.5 family, in 4-bit MLX format) fine-tuned with LoRA to translate natural-language questions into SQL queries. Runs entirely on Apple Silicon Macs using MLX, e.g. starting from 4 GB shared VRAM.

You: "What is the total population of cities in Switzerland (CH)?"
  → SELECT SUM(population) FROM cities WHERE country = 'CH'

Model description

• Base model: mlx-community/Qwen3.5-0.8B-OptiQ-4bit — the compact 0.8B-parameter version, 4-bit quantized (~0.5 GB)
• Fine-tuning: LoRA (no full-weight updates), adapters fused into the base model
• Task: schema-conditioned text-to-SQL generation
• Memory footprint: runs comfortably in ~4 GB of unified memory

The base 0.8B model produces 0% valid SQL out of the box (it answers in prose or hallucinates numbers). Prompt engineering ("Only answer in SQL") lifts this to just 1.5%. After 600 LoRA iterations, the model emits valid SQL in 86.5% of test cases and reaches 47% semantic accuracy (exact-match of query semantics on held-out data).

Intended use cases

• Natural-language querying of small, known schemas (prototypes, local tools, demos)
• On-device / privacy-preserving assistants that turn questions into SQL without sending data anywhere
• Education: studying how LoRA teaches a tiny model a structured output skill
• Embedding a lightweight NL→SQL layer into Mac/iOS apps via MLX

Not intended for: production analytics on large multi-table databases, complex multi-join/window-function SQL, or non-English input.

Usage

Requires mlx-lm:

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("ulldma/Qwen3.5-0.8B-OptiQ-4bit-text-to-sql")

schema = (
    "CREATE TABLE employees (Name VARCHAR, Department VARCHAR, "
    "Salary INT, Start_Date DATE);"
)
prompt = f"{schema}\nQ: Who earns more than 100k in Engineering?\nA: "

response = generate(model, tokenizer, prompt=prompt, max_tokens=100)
print(response)
# SELECT Name FROM employees WHERE Department = 'Engineering' AND Salary > 100000;

Or from the command line:

python -m mlx_lm generate \
  --model ulldma/Qwen3.5-0.8B-OptiQ-4bit-text-to-sql \
  --max-tokens 100 \
  --prompt "CREATE TABLE cities (name VARCHAR, country_code VARCHAR, population INT);
Q: What is the total population of cities in Switzerland (CH)?
A: "

Prompt format

The model expects the exact format used during training:

CREATE TABLE ... ; [INSERT INTO ... VALUES (...);]
Q: <natural language question>
A:

It completes the A: line with a single SQL statement.

Training details

Trained on an Apple Silicon Mac with mlx-lm; see the training repo for the full pipeline (uv sync → train → eval).

Data

Fine-tuned on a filtered subset of gretelai/synthetic_text_to_sql: 5,000 train / 500 validation / 1,000 test examples covering basic SELECT queries and single joins across ~100 domains (healthcare, finance, education, …).

Limitations

• Small model: expect mistakes on complex schemas, aggregations over many tables, or ambiguous questions
• Trained only on basic SQL + single-join complexity — multi-join, nested, and DDL-heavy queries are out of distribution
• Synthetic training data; real-world schema vocabulary may differ
• Semantic accuracy is 47% — always review generated SQL before executing it against real data