llama3.2-3b-sql-expert-1-epoch

Model Card for Llama3.2-3B-SQL-Expert-1Epoch

Model Details

Model Description

Llama3.2-3B-SQL-Expert-1Epoch is a fine-tuned version of Meta’s Llama-3.1-3B, specifically optimized for generating SQL queries from natural language input. The model has been trained using Unsloth for efficient fine-tuning and inference.

• Developed by: Azzedine (GitHub: Azzedde)
• Funded by [optional]: N/A
• Shared by [optional]: Azzedde
• Model Type: Large Language Model (LLM) optimized for SQL query generation
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model [optional]: Meta-Llama-3.1-3B-Instruct

---

Model Sources

• Repository: Hugging Face
• Paper [optional]: N/A
• Demo [optional]: N/A

---

Uses

Direct Use

This model is designed for generating SQL queries based on natural language inputs and is useful for:

• Database management and administration
• Automated query generation
• Data analytics pipelines
• SQL education and training
• Business intelligence applications

Downstream Use [optional]

• Embedding into LLM-based database assistants
• Automating SQL-based analytics
• Assisting developers in writing optimized queries

Out-of-Scope Use

• General NLP tasks unrelated to SQL query generation
• Applications requiring strong factual accuracy outside SQL

---

Bias, Risks, and Limitations

• Incorrect or suboptimal queries: The model may generate queries that are syntactically correct but do not yield the intended results.
• Lack of query optimization: The generated queries are not always optimized for performance; users should validate execution plans.
• English-only support: The model primarily supports English-language inputs.
• Limited schema understanding: The model does not validate database structures and may assume incorrect relationships between tables.

Recommendations

Users should:

• Always validate generated SQL queries before executing them.
• Use the model as an assistant, not a replacement for SQL expertise.
• Fine-tune the model further for domain-specific databases.

---

How to Get Started with the Model

Use the following code to load and use the model:

from unsloth import FastLanguageModel
from transformers import AutoTokenizer

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("Azzedde/llama3.2-3b-sql-expert-1-epoch")
model = FastLanguageModel.from_pretrained("Azzedde/llama3.2-3b-sql-expert-1-epoch")

# Example inference
sql_prompt = """Below is a SQL database schema and a question. Generate an SQL query to answer the question.

### Schema:
{schema}

### Question:
{question}

### SQL Query:
"""
input_text = sql_prompt.format(
    schema="CREATE TABLE employees (id INT PRIMARY KEY, name VARCHAR, salary DECIMAL, department_id INT);",
    question="Find the average salary per department."
)

# Tokenize and generate query
inputs = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)
print(tokenizer.decode(outputs[0]))

---

Training Details

Training Data

• The model was fine-tuned on a structured SQL dataset, including a mix of publicly available SQL benchmarks and synthetically generated SQL queries.

Training Procedure

• Preprocessing: Tokenized using standard SQL syntax formatting
• Training Hyperparameters:
  • batch_size = 4
  • gradient_accumulation_steps = 8
  • num_train_epochs = 1
  • learning_rate = 2e-4
  • fp16 = True

---

Evaluation

Testing Data

• The model was evaluated on a separate test set of SQL queries derived from real-world database schemas.

Evaluation Metrics

• Exact Match Accuracy: Percentage of queries that exactly match ground-truth SQL
• Execution Success Rate: Percentage of generated queries that execute without errors

Results

• High accuracy for common SQL queries
• Some errors in complex multi-table joins and nested queries

---

Environmental Impact

• Hardware Type: Tesla T4 (Google Colab)
• Training Duration: ~1.5 hours
• Compute Region: N/A
• Estimated Carbon Emissions: Minimal

---

Technical Specifications

Model Architecture and Objective

• Based on Llama-3.1 3B, fine-tuned with LoRA for SQL generation.

Compute Infrastructure

• Fine-tuned using Unsloth for efficient training and inference.

Hardware

• GPU: Tesla T4
• Max Reserved Memory: ~6.5 GB

Software

• Libraries Used: unsloth, transformers, TRL, datasets

---

Citation [optional]

BibTeX:

@article{llama3.2-3B-SQL-Expert,
  author    = {Azzedde},
  title     = {Llama3.2-3B-SQL-Expert: An SQL Query Generation Model},
  year      = {2025},
  url       = {https://huggingface.co/Azzedde/llama3.2-3b-sql-expert-1-epoch}
}

APA:

Azzedde. (2025). Llama3.2-3B-SQL-Expert: An SQL Query Generation Model. Retrieved from Hugging Face

---

More Information

For questions, reach out via Hugging Face discussions or GitHub issues.

---

Model Card Authors

• Azzedde (GitHub: Azzedde)

---

Model Card Contact

• Contact: Hugging Face Profile