Llama-3.1-8B-text-to-sql-10K-RussianDataset_Q6_K

Llama3-1-8B fine-tuned with 10K lines of text-to-sql dataset with russian prompts localization

Info

• Developed by: Tvisterious
• License: mit
• Finetuned from model : unsloth/llama-3.1-8b

Model discription and purpose

This middle-size model was trained and quantized to generate SQL commands quickly on middle-end hardware.

Model was fine-tuned with the 10К lines of Tvisterious/gretelai_synthetic_text_to_sql_russian_prompts_localization dataset. It contains more than 80К lines with russian prompts, data base contexts and sql-commands. This is machine-translated origial gretelai/synthetic_text_to_sql dataset, including translation of the database content and filtering parts of sql-commands and containing only SELECT queries. Note that alpaca-prompt was used for fine-tuning. The model has not been tested with prompts in English or other languages, so it may be unstable.

Usage

For using this model you can follow the usage example below for CPU or you can download the gguf-file and use standard llama_cpp functional:

1. CPU

from huggingface_hub import hf_hub_download
from llama_cpp import Llama
model_path = hf_hub_download(
    repo_id="Tvisterious/Llama-3.1-8B-text-to-sql-10K-RussianDataset_Q6_K",
    filename="Llama-3.1-8B-text-to-sql-10K-RussianDataset_Q6_K.gguf",
    cache_dir="./models"
)
llm = Llama(
    model_path=model_path,
    n_ctx=1024,
    n_threads=8
)
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
        ### Instruction:
        SQL Prompt: {}
        ### Input:
        Company database: {}
        ### Response:
        SQL: {}
        """
response = llm(
    alpaca_prompt.format(
                "Сколько есть работников с красными машинами?", # instruction 'How many workers have red cars?'
                "T_Workers(worker_id, name, age, id_car), T_Cars(car_id, mark, type, color)", # input with DB context
                "", # output - leave this blank for generation!
                ),
    max_tokens=256,
    temperature=0.7
)
print(response['choices'][0]['text'])