llama-3-sqlcoder-8b-GGUF

llama-3-sqlcoder-8b — GGUF

GGUF (llama.cpp) build of defog/llama-3-sqlcoder-8b, a Llama-3 8B model fine-tuned for text-to-SQL generation. This repo packages a Q4_K_M quantization so the model runs efficiently on CPU/GPU through llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes.

Files

Usage

llama.cpp

./llama-cli -m llama-3-sqlcoder-8b.Q4_K_M.gguf -p "Generate a SQL query to answer the question."

Ollama (Modelfile)

FROM ./llama-3-sqlcoder-8b.Q4_K_M.gguf

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="omeryentur/llama-3-sqlcoder-8b-GGUF",
    filename="llama-3-sqlcoder-8b.Q4_K_M.gguf",
)
out = llm("### Task\nGenerate a SQL query to answer the question.\n### Question\nHow many users signed up in 2024?\n### SQL\n")
print(out["choices"][0]["text"])

Use the prompt format expected by the base defog/llama-3-sqlcoder-8b model for best results.

Credits

• Base model: defog/llama-3-sqlcoder-8b
• Quantization tooling: llama.cpp

See more text-to-SQL work on this profile, including the Text-to-PostgreSQL dataset.