matsuo-llm-advanced-dbbench-v1

DBBench SQL Agent - Phase 1 (Multi-Turn Fine-Tuned)

Model Description

This is a fine-tuned Qwen2.5-7B-Instruct model optimized for DBBench SQL agent tasks in the Matsuo Lab LLM Course 2025 Advanced Competition.

Version: Phase 1 (DBBench specialized) Key Innovation: Multi-turn trajectory training with 3 patterns (direct, exploration, self-correction)

Training Details

• Base Model: Qwen/Qwen2.5-7B-Instruct
• Training Method: QLoRA (4-bit quantization + LoRA, merged)
• LoRA Config: r=32, alpha=32, dropout=0, target_modules=[q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]
• Dataset: 7,490 multi-turn SQL trajectories (Spider train + BIRD mini_dev)
• Training: 0.5 epoch, lr=5e-5, effective batch size=16
• Final Train Loss: 0.22
• Eval Loss: 0.235
• Training Time: ~10 min (RTX 5090 + Unsloth)

Data Generation

Training data was generated using rule-based methods only (no LLM involvement):

• Source: Spider train split (6,990 samples) + BIRD mini_dev (500 samples)
• Patterns: Direct answer (40%), Exploration (30%), Self-correction (30%)
• Schema: Auto-generated CREATE TABLE from SQL parsing (sqlparse)
• Results: Template-based fake MySQL CLI output
• Reasoning: Template-based English text (random.choice from predefined strings)
• No LLM used for data generation, filtering, or quality assessment

Response Format

The model follows the DBBench protocol:

For SQL queries:

[Reasoning about the question]

Action: Operation
```sql
SELECT column FROM table WHERE condition;

**For final answers:**

[Reasoning about the result]

Action: Answer Final Answer: ["ANSWER1", "ANSWER2", ...]

## Local Test Results (5/5 Passed)

All 5 test queries produced valid format with:
- Correct `Action: Operation` + SQL block structure
- Single-line SQL statements
- MySQL-compatible syntax with backtick quoting
- Natural reasoning text before each action

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "astom-M/matsuo-llm-advanced-dbbench-v1"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="auto",
    torch_dtype="bfloat16"
)

Competition

Event: Matsuo Lab LLM Course 2025 Advanced Competition Target: DBBench task improvement (baseline 51.3% → target 60-70%) Training Date: February 2026

License

Same as base model (Qwen2.5-7B-Instruct) - Apache 2.0