Qwen3-4B-DBT-Instruct-LoRA

Qwen3-4B-DBT-Instruct — LoRA Adapter

QLoRA adapter fine-tuned on top of unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit (the pre-quantized 4-bit Unsloth variant of Qwen3-4B-Instruct) to convert natural language business questions into complete, multi-file dbt DAGs.

Looking for the ready-to-run GGUF (LM Studio / Ollama)? → tdelard/Qwen3-4b-DBT-Instruct-GGUF

---

What does this model do?

Given a business question and a SQL schema, the model generates a full dbt project structure:

• Staging layer — stg_*.sql files that clean and rename raw source data
• YAML sources — _sources.yml / _stg_*.yml schema files with column definitions
• Intermediate models — int_*.sql files that join and enrich staging data
• Marts layer — fct_*.sql or dim_*.sql final business models

---

Usage

This repo contains only the LoRA adapter weights (264 MB). You need to load the base model alongside the adapter.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit"
adapter_id    = "tdelard/Qwen3-4B-DBT-Instruct-LoRA"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model     = AutoModelForCausalLM.from_pretrained(base_model_id, torch_dtype="auto", device_map="auto", load_in_4bit=True)
model     = PeftModel.from_pretrained(model, adapter_id)

messages = [
    {
        "role": "system",
        "content": (
            "You are a dbt expert. Given a business question and a SQL schema, "
            "generate a complete, production-ready dbt DAG including staging SQL files, "
            "YAML schema files, intermediate models, and mart models. "
            "Use proper dbt conventions: ref(), source(), naming prefixes (stg_, int_, fct_, dim_)."
        ),
    },
    {
        "role": "user",
        "content": (
            "Business question: Show the total revenue per product category.\n"
            "SQL context: CREATE TABLE orders (order_id INT, product_id INT, amount DECIMAL); "
            "CREATE TABLE products (product_id INT, category VARCHAR, name VARCHAR);"
        ),
    },
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=0.7, top_p=0.8, top_k=20)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

---

Training details

---

Source code

Full training pipeline: thomasdlr/qwen3-dbt-instruct

---

License

Apache 2.0 — same as the base Qwen3-4B / Unsloth model.

---

Fine-tuned 2x faster using Unsloth.