llama-3-sqlcoder-8b-exl2

Exllama v2 defog/llama-3-sqlcoder-8b

Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.0.21">turboderp's ExLlamaV2 v0.0.21</a> for quantization.

<b>The "main" branch only contains the measurement.json, download one of the other branches for the model</b>

Each branch contains an individual bits per weight, with the main one containing only the meaurement.json for further conversions.

Original model: <a href="https://huggingface.co/defog/llama-3-sqlcoder-8b">defog/llama-3-sqlcoder-8b</a><br> Calibration dataset is based on: <a href="https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction">m-a-p/CodeFeedback-Filtered-Instruction</a>

Available sizes

A capable language model for text to SQL generation for Postgres, Redshift and Snowflake that is on-par with the most capable generalist frontier models.

Model Description

Developed by: Defog, Inc Model type: [Text to SQL] License: [CC-by-SA-4.0] Finetuned from model: [Meta-Llama-3-8B-Instruct]

Demo Page

https://defog.ai/sqlcoder-demo/

Ideal prompt and inference parameters

Set temperature to 0, and do not do sampling.

Prompt

<|begin_of_text|><|start_header_id|>user<|end_header_id|>

Generate a SQL query to answer this question: `{user_question}`
{instructions}

DDL statements:
{create_table_statements}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

The following SQL query best answers the question `{user_question}`:
```sql

Evaluation

This model was evaluated on SQL-Eval, a PostgreSQL based evaluation framework developed by Defog for testing and alignment of model capabilities.

You can read more about the methodology behind SQLEval here.

Contact

Contact us on X at @defogdata, or on email at founders@defog.ai