The_Scamper-opt70bqt

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

• Developed by: The Scamper
• Model type: Transformer
• Language(s) (NLP): Thai, English
• License: apache-2.0
• Finetuned from model: OpenThaiGPT-1.0.0 70B (https://huggingface.co/openthaigpt/openthaigpt-1.0.0-70b-chat)

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

The Tubular Question Answering Large Language Model is based on OpenThaiGPT and fine-tuned for converting natural language questions into SQL queries. It learns to map the nuances of Thai language to SQL structures, enabling efficient retrieval of information from databases.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

How to Get Started with the Model

Use the code below to get started with the model.

>>> model2_path ="AIAT/The_Scamper-opt70bqt"
>>> tokenizer = AutoTokenizer.from_pretrained(model2_path, padding_side="right",use_fast=False)
>>> model = AutoModelForCausalLM.from_pretrained(model2_path,
                                             device_map="auto")

Training Details

Training Data

Dataset: https://huggingface.co/datasets/AIAT/The_Scamper-train

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

The methodology for fine-tuning involves a dataset with three columns: "instruction", "question" and "SQL syntax". Here's a brief outline of the process:

1. Data Collection: Gather a dataset containing pairs of questions and their corresponding SQL queries. Ensure the questions cover various topics and query types, while the SQL queries represent the desired actions on a database.

2. Pre-processing: Clean and preprocess the data to remove noise, standardize formatting, and handle any inconsistencies. Tokenize the text and encode it into a format suitable for training.

3. Model Architecture: Utilize OpenThaiGPT 1.0.0 70B as the base model.

4. Fine-tuning Setup: Divide the dataset into training (90%) and test sets (10%). We define the training procedure, including hyperparameters such as learning rate, batch size, and number of training epochs.

5. Fine-tuning Process: Train the model on the question-SQL pairs using the defined setup. During training, the model learns to predict the SQL query corresponding to a given question by minimizing a suitable loss function.

6. Testing: Evaluate the final model on a held-out test set to assess its generalization performance on unseen data.

By following this methodology, the model can be fine-tuned to accurately convert natural language questions into SQL syntax, enabling seamless interaction with structured databases.