llama-3-8b-text-to-ddl-lora

LLaMA-3 8B Text-to-SQL (Schema Architect) 🏗️

This model is a fine-tuned version of Meta's LLaMA-3 8B, specialized in generating complex SQL Data Definition Language (DDL) scripts from natural language descriptions.

It has been trained to act as a "Strict SQL Architect" that can dynamically switch between Normalized (3NF) and Denormalized (Transaction Log) schema designs based on specific instructions.

Model Details

• Base Model: unsloth/llama-3-8b-bnb-4bit
• Fine-tuning Framework: Unsloth (LoRA Adapters)
• Task: Text-to-SQL (DDL Generation)
• License: Apache-2.0
• Language: English

Key Features

• Dual-Mode Generation: Can generate standard normalized schemas OR denormalized transaction logs based on the instruction.
• Explicit Parsing: Trained to understand "robotic" and explicit descriptions of database entities (e.g., "The database consists of...").
• Constraint Adherence: Strictly follows instructions to avoid hallucinating columns that are not explicitly mentioned in the input.

Installation

Since this model contains LoRA adapters, you need to load the base model and apply these adapters. We recommend using unsloth for 2x faster inference, but standard peft works too.

Option 1: Using Unsloth (Recommended)

pip install unsloth

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "YOUR_HF_USERNAME/llama-3-8b-text-to-sql-lora", # Replace with your Repo ID
    max_seq_length = 2048,
    dtype = None,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

Option 2: Using Standard Transformers & PEFT

pip install transformers peft bitsandbytes

from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

config = PeftConfig.from_pretrained("YOUR_HF_USERNAME/llama-3-8b-text-to-sql-lora")
base_model = AutoModelForCausalLM.from_pretrained("unsloth/llama-3-8b-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "YOUR_HF_USERNAME/llama-3-8b-text-to-sql-lora")
tokenizer = AutoTokenizer.from_pretrained("YOUR_HF_USERNAME/llama-3-8b-text-to-sql-lora")

How to Use (Prompt Template)

CRITICAL: This model was fine-tuned on a specific prompt format. You must use the template below for optimal results. The input text should be descriptive and explicit about the entities.

Prompt Template

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{INSTRUCTION}

### Input:
{INPUT_DESCRIPTION}

### Response:

Example 1: Normalized Schema (Standard)

Instruction:

Design a standard normalized SQL schema for this scenario. Use standard many-to-many relationship tables to ensure data integrity without redundancy.

Input:

Hospital Management System. The database consists of the following tables: Table Patient has columns: Patient_ID, Name, DOB, Insurance_Provider. Table Doctor has columns: Doctor_ID, Name, Specialty, Years_Exp. Table Appointment has columns: Appt_ID, Date, Time, Room_Number, Patient_ID, Doctor_ID. Foreign key relationships: Table Appointment contains a foreign key references table Patient. Table Appointment contains a foreign key references table Doctor.

Example 2: Denormalized Schema (Transaction Log)

Instruction:

Design a denormalized SQL schema using a Transaction table. You MUST allow for historical repetition and duplicate entries. Do NOT use standard relationship tables.

Input:

Hospital Management System. The database consists of the following tables: Table Patient has columns: Patient_ID, Name, DOB, Insurance_Provider. Table Doctor has columns: Doctor_ID, Name, Specialty, Years_Exp. Table Transaction_Log has columns: Transaction_ID, Visit_Date, Room_Number, Procedure_Code, Billed_Amount, Patient_ID, Doctor_ID. Foreign key relationships: Table Transaction_Log contains a foreign key references table Patient. Table Transaction_Log contains a foreign key references table Doctor.

Training Data

The model was fine-tuned on a high-quality dataset consisting of ~500 pairs of Normalized vs. Denormalized schema scenarios.

• Core Data: Includes 18 original, human-curated scenarios that served as high-quality seed templates.
• Synthetic Expansion: The remaining examples were synthetically generated by Gemini Pro to strictly follow the logic of the original templates across diverse domains like Healthcare, Supply Chain, IoT, and FinTech.
• Quality Control: All data was strictly curated to ensure the "Input" text explicitly matches the "Output" columns (Lossless inputs) to prevent the model from learning to hallucinate attributes.

Evaluation

Achieved 100% semantic accuracy on a held-out test set of complex schema scenarios, correctly identifying table structures, column types, and foreign key relationships in both normalized and denormalized modes.

---

Created by: [MohabMostafa] Fine-tuned with: Unsloth