math_QA-Mistral-7B-QLoRA-adapter

This repo contains LoRA adapter weights.

Model Description

• Project GitHub Page: https://github.com/adityasihag1996/math_QA.git
• Developed by: Aditya Sihag
• Model type: fine-tuned using QLoRA on 1x RTX 4090
• Finetuned from model: mistralai/Mistral-7B-v0.1

Results

<table> <thead> <tr> <th>Prompt Approach</th> <th>GSM8k</th> <th>MATH</th> </tr> </thead> <tbody> <tr> <td>Zero-Shot CoT</td> <td><b>75.81</b></td> <td><b>-</b></td> </tr> </tbody> </table>

Training procedure

The following bitsandbytes quantization config was used during training:

• quant_method: bitsandbytes
• load_in_8bit: False
• load_in_4bit: True
• bnb_4bit_quant_type: nf4
• bnb_4bit_use_double_quant: True
• bnb_4bit_compute_dtype: float16

LoraConfig params:

• r: 128
• lora_alpha: lora_r * 2
• lora_dropout: 0.05
• bias: "none"
• task_type: "CAUSAL_LM"
• target_modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]

The hyperparameters for the LoRA fine-tuning are listed below:

• epochs: 3
• learning_rate: 5e-5
• batch_size: 256
• max_grad_norm: 1.0
• weight_decay: 0.001
• lr_scheduler_type: "cosine"
• warmup_ratio: 0.03

Dataset

math_QA dataset is prepared as combination of MetaMathQA and MathInstruct, and some internal data. Refer math_QAaugP

Model Usage

import torch
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer
)
from peft import PeftModel

model_path = "mistralai/Mistral-7B-v0.1"
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype = torch.float16,
    device_map = {"": 0},
)

# Load LoRA and merge
model = PeftModel.from_pretrained(model, "adityasihag/math_QA-Mistral-7B-QLoRA-adapter")
model = model.merge_and_unload()

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token

question = """Solve the linear equations. $3(x+2)-x=x + 9$. Find the value of x."""

sample_input = f"""Question: {question} \n Answer: """

sample_input_tokenised = tokenizer(sample_input, return_tensors = "pt").to("cuda")

generated_ids = model.generate(
                    **sample_input_tokenised,
                    max_new_tokens = 1024,
                    temperature = 0.3
                )
output = tokenizer.decode(generated_ids[0], skip_special_tokens = True)
print(output)

Sample Input:

Question: Solve the linear equations. $3(x+2)-x=x + 9$. Find the value of x. \n Answer: 

Model Output:

Given the linear equation 3(x+2)-x=x+9. 
First, distribute the 3 in the brackets to get 3x + 6 - x = x + 9. 
Simplify the equation to get 2x + 6 = x + 9. 
Next, transpose x from the right side to the left side and from the left side to the right side to get x = 9 - 6. 
Finally, solve for x to get x = 3.

Prompt Template:

Question: <question>
Answer: 

Comparing math_QA models with other SFT LLM models