Qwen2-complex-UK

Model Card: Qwen2-complex-UK

Model Description

Qwen2-complex-UK is a fine-tuned large language model specialized in solving complex mathematical problems, particularly those involving symbolic manipulation, function evaluation, and algebraic expressions. It is built upon the powerful Qwen2-Math-1.5B base model by Qwen, further enhancing its capabilities for precise and step-by-step mathematical reasoning.

This model was fine-tuned using the ujjwal52/Maths-medium-7k-COT-Uk dataset, which provides a rich collection of mathematical problems with Chain-of-Thought (COT) explanations. This allows the model to not only provide correct answers but also to articulate the reasoning process, making it highly suitable for educational applications, research, and any task requiring interpretable mathematical solutions.

Why Choose This Model?

1. Enhanced Mathematical Reasoning: While the base Qwen2-Math-1.5B is already proficient, this fine-tuned version excels in handling more complex and nuanced mathematical queries. The specific dataset used focuses on medium-difficulty problems, ensuring robust performance across a range of mathematical challenges.

2. Chain-of-Thought (COT) Capabilities: Thanks to the training dataset, Qwen2-complex-UK can generate detailed, logical steps to arrive at a solution. This is invaluable for users who need to understand how an answer was derived, not just what the answer is.

3. Efficiency with QLoRA: Fine-tuned using QLoRA with 4-bit quantization, this model achieves impressive performance while maintaining a relatively small footprint, making it accessible for deployment in environments with limited resources.

4. Versatile Applications: Ideal for automated math problem solvers, intelligent tutoring systems, content generation for math education, and assisting researchers with complex calculations.

Training Details

Base Model

• Qwen/Qwen2-Math-1.5B

Fine-tuning Method

• QLoRA (Quantized LoRA): Efficient fine-tuning technique that reduces memory usage during training.

• 4-bit Quantization: Model weights loaded in 4-bit NormalFloat (NF4) precision for memory efficiency.

Dataset

• ujjwal52/Maths-medium-7k-COT-Uk: A specialized dataset containing 7,000 medium-difficulty mathematical problems with detailed Chain-of-Thought (COT) explanations

How to Use

This model can be easily loaded and used for text generation tasks, particularly for mathematical problem-solving. Here's how you can use it with the transformers library:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

# Define the model ID on Hugging Face
model_id = "ujjwal52/Qwen2-complex-UK"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Load model (make sure to specify the correct dtype and device_map for your setup)
# If you saved it with 4-bit, you might need BitsAndBytesConfig for inference if not merged fully
# For the merged model, float16 is usually appropriate.
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,  # Use the dtype the merged model was saved in
    device_map="auto",           # Automatically maps model to available devices (e.g., GPU)
)

# Create a text generation pipeline
prompt = "If $f(x) = \frac{3x-2}{x-2}$, what is the value of $f(-2) +f(-1)+f(0)$? Express your answer as a common fraction.,ans carefully"
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=3500)

# Format the prompt according to the Llama 2 chat template (or Qwen2's equivalent)
# The fine-tuning typically uses a specific template, here we assume a chat-like structure.
chat_template_prompt = f"<s>[INST] {prompt} [/INST]"

# Generate text
result = pipe(chat_template_prompt)
print(result[0]['generated_text'])

# Another example:
prompt_2 = "Solve the equation: $2x + 5 = 11$. Explain your steps."
chat_template_prompt_2 = f"<s>[INST] {prompt_2} [/INST]"
result_2 = pipe(chat_template_prompt_2)
print(result_2[0]['generated_text'])

Limitations and Biases

• Mathematical Domain: While strong in algebra and function evaluation, the model's performance might vary in other highly specialized mathematical fields (e.g., advanced topology, quantum mechanics) not extensively covered in its training data.
• Problem Complexity: The model was fine-tuned on medium-difficulty problems. Very simple or extremely complex, open-ended mathematical research questions might still pose a challenge.
• Hallucination: Like all large language models, it can sometimes generate plausible-sounding but incorrect information. Always verify critical mathematical results.
• Language Bias: The dataset (ujjwal52/Maths-medium-7k-COT-Uk) influences the model's style and terminology. Be aware of potential biases in how problems are framed or solved.

License

This model is based on Qwen2-Math-1.5B, which is typically under a specific license (e.g., Apache 2.0 or similar permissive license). Please refer to the Qwen2-Math-1.5B model page for its exact licensing terms. The fine-tuned weights inherit the base model's license.

Citation

If you use this model in your research or application, please consider citing the original Qwen2 work and the dataset used for fine-tuning:

@misc{qwen2,
    title={Qwen2: A New Series of Large Language Models}, 
    author={The Qwen Team},
    year={2024},
    publisher={Hugging Face},
    url={https://huggingface.co/Qwen/Qwen2-Math-1.5B}
}

@misc{maths_medium_7k_cot_uk,
    title={Maths-medium-7k-COT-Uk Dataset},
    author={ujjwal52},
    year={2024},
    publisher={Hugging Face},
    url={https://huggingface.co/datasets/ujjwal52/Maths-medium-7k-COT-Uk}
}