ecu-pilot-qwen3-8b-GGUF

ecu-pilot-qwen3-8b-GGUF

A fine-tuned Qwen3-8B model for dbt project assistance via tool calling against dbt-index.

Overview

ecu-pilot is trained to autonomously select and call the correct dbt-index tool (out of 9 available tools) based on natural language questions about a dbt project, then summarize the results concisely.

Tools

Training

• Base model: Qwen3-8B (via unsloth/Qwen3-8B-bnb-4bit)
• Method: QLoRA (r=32, alpha=32) with Unsloth + TRL SFTTrainer
• Data: 667 tool-calling conversation examples from 8 dbt projects (BIRD benchmark databases)
• Epochs: 1
• Hardware: AWS EC2 g6 (NVIDIA L40S 48GB)

Quantization

• Format: GGUF Q4_K_M (4-bit quantized via llama.cpp)
• Size: ~4.7 GB

Chat Template

This model uses a custom chat template for tool calling. The template is included as chat_template.jinja in this repo.

Key differences from stock Qwen3:

• Tool definitions are injected into the system message within <tools> XML tags
• Tool calls use <tool_call> / </tool_call> XML delimiters
• Tool responses are wrapped in <tool_response> / </tool_response> and sent as user messages

When using with inference servers that support Jinja templates (vLLM, TGI, llama.cpp server), point to this template file.

Usage

LM Studio

1. Download the .gguf file
2. Place in ~/.lmstudio/models/json-l/ecu-pilot-qwen3-8b-GGUF/
3. Load in LM Studio, set the chat template to the contents of chat_template.jinja, and start the local server

Ollama

Create a Modelfile:

FROM ./ecu-pilot-qwen3-8b-q4km.gguf

TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{- if .Tools }}<|im_start|>system
# Tools
You are provided with function signatures within <tools></tools> XML tags:
<tools>
{{- range .Tools }}
{{ json . }}
{{- end }}
</tools>
<|im_end|>
{{ end }}{{- range .Messages }}<|im_start|>{{ .Role }}
{{- if .Content }}
{{ .Content }}
{{- end }}{{- if .ToolCalls }}
<tool_call>
{{- range .ToolCalls }}
{"name": "{{ .Function.Name }}", "arguments": {{ json .Function.Arguments }}}
{{- end }}
</tool_call>
{{- end }}<|im_end|>
{{ end }}<|im_start|>assistant
"""

SYSTEM """You are an expert dbt project assistant with access to a dbt-index metadata server."""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<tool_call>"
PARAMETER temperature 0.7
PARAMETER num_ctx 4096

Then:

ollama create ecu-pilot -f Modelfile
ollama run ecu-pilot

Evaluation

On 8 test prompts (2 runs each):

License

Apache 2.0