Llama3-2-3B-Instruct-text-to-sql-20K

Llama3-2-3B fine-tuned with 20K lines of text-to-sql dataset with russian prompts localization

Info

• Developed by: Tvisterious
• License: apache-2.0
• Finetuned from model : unsloth/llama-3.2-3b-instruct-unsloth-bnb-4bit

Model discription and purpose

This small model was trained to generate SQL commands quickly, even on slow hardware:

• The speed using CUDA with T4 GPU is 1-3 sec.
• The speed on Intel N100 CPU with 16Gb RAM is about 40-60 sec.

Model was fine-tuned with the 20К lines of Tvisterious/gretelai_synthetic_text_to_sql_russian_prompts_localization dataset. It contains more than 80К lines with russian prompts, data base contexts and sql-commands. This is machine-translated origial gretelai/synthetic_text_to_sql dataset, including translation of the database content and filtering parts of sql-commands and containing only SELECT queries. Note that alpaca-prompt was used for fine-tuning. The model has not been tested with prompts in English or other languages, so it may be unstable.

Usage

For using this model you can follow the usage example below both for CPU and GPU:

1. CPU

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
      "Tvisterious/Llama3-2-3B-Instruct-text-to-sql-20K",
      torch_dtype=torch.float16,
      device_map="cpu",
      low_cpu_mem_usage=True)

tokenizer = AutoTokenizer.from_pretrained("Tvisterious/Llama3-2-3B-Instruct-text-to-sql-20K")

alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
SQL Prompt: {}

### Input:
Company database: {}

### Response:
SQL: {}"""

prompt_text = alpaca_prompt.format(
                "Сколько есть работников с красными машинами?", # instruction 'How many workers have red cars?'
                "T_Workers(worker_id, name, age, id_car), T_Cars(car_id, mark, type, color)", # input with DB context
                "" # output - free space for generation!
            )

inputs = tokenizer(prompt_text, return_tensors="pt")
outputs = model.generate(
          inputs.input_ids,
          max_new_tokens=150,
          temperature=0.4,
          do_sample=True
          )
            
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

1. GPU

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Tvisterious/Llama3-2-3B-Instruct-text-to-sql-10K",
    torch_dtype=torch.float16,
).to('cuda')

tokenizer = AutoTokenizer.from_pretrained("Tvisterious/Llama3-2-3B-Instruct-text-to-sql-20K")

alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
SQL Prompt: {}

### Input:
Company database: {}

### Response:
SQL: {}"""

prompt_text = alpaca_prompt.format(
                "Сколько есть работников с красными машинами?", # instruction 'How many workers have red cars?'
                "T_Workers(worker_id, name, age, id_car), T_Cars(car_id, mark, type, color)", # input with DB context
                "" # output - free space for generation!
            )

inputs = tokenizer(prompt_text, return_tensors="pt").to('cuda')
outputs = model.generate(
          inputs.input_ids,
          max_new_tokens=150,
          temperature=0.4,
          do_sample=True
          )
            
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth