LogicFlow-Gemma-3-27b-thinking

LogicFlow-Gemma-3-27b-thinking

Model Description

LogicFlow-Gemma-3-27b-thinking is an advanced multimodal reasoning model built upon google/gemma-3-27b-it, specifically designed to excel at complex logical reasoning, mathematical problem-solving, and step-by-step analytical thinking. This model represents a significant advancement in AI reasoning capabilities, achieved through careful fine-tuning on three specialized, high-quality datasets using LoRA (Low-Rank Adaptation) technique.

Key Innovations

This unique combination of datasets creates a model that not only provides correct answers but also demonstrates how it arrives at those answers, making it particularly valuable for educational applications, research, and any scenario requiring explainable AI reasoning.

The model demonstrates enhanced capabilities in:

• Logical Reasoning: Improved ability to work through complex logical problems step by step
• Mathematical Problem Solving: Enhanced performance on mathematical reasoning tasks (76.8% MATH, 13.3% AIME25)
• Scientific Analysis: Exceptional scientific reasoning capabilities (45.96% GPQA Diamond)
• Chain-of-Thought Reasoning: Superior step-by-step thinking with detailed reasoning chains and self-verification
• Structured Analysis: Improved at breaking down complex problems into manageable components
• Multi-Method Verification: Uses multiple approaches to validate results and ensure accuracy
• Vision Understanding: Ability to analyze and reason about images, charts, diagrams, and visual data
• Multimodal Reasoning: Combining visual and textual information for comprehensive analysis

Model Details

• Model Type: Multimodal Language Model (Gemma-3 Architecture)
• Base Model: google/gemma-3-27b-it
• Parameters: 27 billion parameters
• Fine-tuning Method: LoRA (Low-Rank Adaptation) with merge
• Context Length: 131,072 tokens
• Architecture: Gemma-3 with vision capabilities
• Precision: bfloat16
• Image Resolution: 896x896 pixels, encoded to 256 tokens per image
• Supported Formats: Text + Images (JPEG, PNG, WebP)

Training Details

Training Data

The model was fine-tuned on three carefully selected, high-quality datasets that form the foundation of its exceptional reasoning capabilities:

OpenO1-SFT Dataset

• Purpose: Supervised fine-tuning for advanced reasoning patterns
• Content: High-quality reasoning demonstrations with explicit thought processes
• Impact: Enables the model to break down complex problems systematically and show transparent reasoning chains

Open-Thoughts Dataset

• Purpose: Step-by-step thinking process modeling
• Content: Detailed internal monologues and reasoning progressions for various problem types
• Impact: Teaches the model to externalize its thinking process, making reasoning transparent and verifiable

OpenR1-Math Dataset

• Purpose: Mathematical reasoning and problem-solving specialization
• Content: Comprehensive mathematical problems with detailed solution methodologies
• Impact: Significantly enhances performance on mathematical reasoning tasks, from basic arithmetic to advanced competition-level problems

This synergistic combination creates a model that excels not only at providing accurate answers but also at demonstrating clear, verifiable reasoning processes.

Training Configuration

Core Training Parameters

• Learning Rate: 5e-05
• Epochs: 5.0
• Optimizer: AdamW (adamw_torch)
• LR Scheduler: Cosine with 100 warmup steps
• Max Gradient Norm: 1.0
• Max Samples: 100,000
• Precision: bfloat16 (bf16: true)

Batch Configuration

• Per Device Train Batch Size: 2
• Gradient Accumulation Steps: 8
• Total Effective Batch Size: 32
• Packing: Disabled (false)

LoRA Configuration

• Fine-tuning Type: LoRA
• LoRA Rank (r): 8
• LoRA Alpha: 16
• LoRA Dropout: 0.0
• LoRA Target: all (comprehensive layer targeting)

Sequence and Vision Parameters

• Cutoff Length: 2,048 tokens
• Image Max Pixels: 589,824
• Image Min Pixels: 1,024
• Video Max Pixels: 65,536
• Video Min Pixels: 256
• Flash Attention: auto
• Freeze Vision Tower: true
• Freeze Multi-modal Projector: true

Special Features

• Template: gemma (Optimized for multimodal reasoning tasks)
• Trust Remote Code: true (Required for advanced vision capabilities)
• Preprocessing Workers: 16 (Optimized for multimodal data processing)
• Save Steps: 100 (Frequent checkpointing for training stability)
• Logging Steps: 5 (Detailed training monitoring)

Training Results

Training Loss Curve

The model training included comprehensive loss tracking and visualization. The training loss curve below shows the convergence pattern over the 41,400 training steps across 5 epochs:

The loss curve demonstrates stable convergence with the final training loss reaching 0.003759, indicating effective learning without overfitting.

Benchmark Performance

Comprehensive Evaluation Results

Key Performance Insights

Significant Improvements

• Mathematical Reasoning: Exceptional improvements - GSM8K (+6.9%) and MATH (+26.8%) demonstrate enhanced step-by-step problem solving
• Advanced Mathematics: Massive 26.8% improvement on MATH benchmark showcases superior mathematical reasoning capabilities
• Scientific Reasoning: Outstanding 45.96% accuracy on GPQA Diamond - significantly above typical model performance (30-35%)
• Competition Mathematics: Solid 13.3% performance on AIME25 - competing with leading models on elite mathematical competitions
• Code Generation: 3.4% improvement on MBPP shows better programming logic understanding
• Domain-Specific Knowledge: Improvements in STEM (+1.6%), Humanities (+2.2%), and Social Sciences (+2.3%)

Trade-offs Observed

• Instruction Following: Slight decrease in IFEval scores (-5% prompt-level, -4.9% instruction-level)
• General Knowledge: Overall MMLU score decreased by 3.3% due to reasoning specialization
• Reasoning Focus: Model optimized for deep analytical thinking over rapid instruction compliance

Specialized Capabilities

• Mathematical Excellence: Outstanding 76.8% accuracy on MATH benchmark - among the top performances for 27B models
• Scientific Reasoning: Exceptional 45.96% on GPQA Diamond - handling graduate-level physics, chemistry, and biology problems
• Elite Competition Performance: Competitive 13.3% on AIME25 - tackling American Invitational Mathematics Exam challenges
• Chain-of-Thought Mastery: Demonstrates sophisticated reasoning through detailed thinking processes with multi-method verification
• Transparent Reasoning: Shows complete work and self-validates answers using multiple approaches (as shown in CoT examples)
• Cross-Domain Expertise: Superior performance spanning mathematics, natural sciences, and logical reasoning

Benchmarking Methodology

Our evaluation follows rigorous benchmarking principles:

1. Reproducible Environment: All tests conducted with fixed random seeds and controlled temperature settings
2. Diverse Metrics: Beyond accuracy, we evaluate reasoning quality, step-by-step explanations, and cross-domain scientific performance
3. Research-Relevant Tasks: Focus on real-world applications in education, scientific research, and advanced technical analysis
4. Comparative Baselines: Direct comparison with original Gemma-3-27B-IT and established benchmarks

Performance Analysis

According to (Domino AI's benchmarking guidelines), we evaluated both predictive characteristics and operational constraints:

• Mathematical & Scientific Excellence: 76.8% MATH accuracy and 45.96% GPQA Diamond represent breakthrough reasoning capabilities
• Competition-Level Performance: 13.3% AIME25 accuracy demonstrates capability in elite mathematical competitions
• Industry Recognition: Based on Google's Gemma 3 announcement, the 27B model achieves 1338 Elo on Chatbot Arena
• Advanced Problem Solving: GPQA Diamond performance significantly exceeds typical model benchmarks (30-35% baseline)
• Latency: Average inference time increased by ~15% due to enhanced reasoning processes - worthwhile trade-off for quality
• Quality: Exceptional improvements in explanation quality - mathematical (+26.8%) and scientific reasoning (+11-16%)
• Reliability: Consistent performance across multiple evaluation runs with detailed step-by-step reasoning chains
• Cross-Domain Specialization: Superior performance in mathematics, natural sciences, and complex logical reasoning

Usage

Installation

For multimodal functionality, ensure you have the latest versions of the required packages:

pip install -U transformers torch torchvision
pip install -U pillow requests
# For GPU acceleration
pip install -U accelerate

Basic Text Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "RekklesAI/LogicFlow-Gemma-3-27b-thinking"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Example usage for reasoning tasks
prompt = """Solve this step by step:
If a train travels 120 km in 2 hours, and then 180 km in the next 3 hours, what is its average speed for the entire journey?

Let me think through this step by step:"""

inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=True,
        top_p=0.95,
        top_k=64,
        temperature=0.7
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Multimodal Usage (Text + Image)

from transformers import AutoProcessor, Gemma3ForConditionalGeneration
from PIL import Image
import requests
import torch

# Load model and processor
model_name = "RekklesAI/LogicFlow-Gemma-3-27b-thinking"
model = Gemma3ForConditionalGeneration.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_name)

# Load an image (example: a mathematical diagram or chart)
url = "https://example.com/math-diagram.jpg"
image = Image.open(requests.get(url, stream=True).raw)

# Create a multimodal prompt for step-by-step analysis
prompt = """<start_of_image>Analyze this mathematical diagram step by step. 
What mathematical concepts are being illustrated, and how would you solve any problems shown?

Please provide a detailed, step-by-step explanation."""

# Process the inputs
model_inputs = processor(text=prompt, images=image, return_tensors="pt")

# Generate response
input_len = model_inputs["input_ids"].shape[-1]
with torch.inference_mode():
    generation = model.generate(
        **model_inputs,
        max_new_tokens=1024,
        do_sample=True,
        top_p=0.95,
        temperature=0.7
    )
    generation = generation[0][input_len:]

# Decode the response
response = processor.decode(generation, skip_special_tokens=True)
print(response)

Chat Template Usage

This model uses the standard Gemma 3 multimodal chat template with optimized formatting:

Text-only Chat

messages = [
    {"role": "system", "content": "You are a helpful AI assistant specialized in logical reasoning and mathematics."},
    {"role": "user", "content": "Explain the reasoning behind the Pythagorean theorem and provide a step-by-step proof."}
]

input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=True,
    top_p=0.95,
    temperature=0.7
)

response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
print(response)

Multimodal Chat (with Images)

from PIL import Image

# Load an image
image = Image.open("path/to/your/image.jpg")

messages = [
    {
        "role": "user", 
        "content": "Analyze this chart and explain the trends you observe. What mathematical relationships can you identify?",
        "images": [image]  # Include image in the message
    }
]

# Use processor for multimodal inputs
model_inputs = processor.apply_chat_template(
    messages, 
    add_generation_prompt=True, 
    return_tensors="pt"
)

outputs = model.generate(
    **model_inputs,
    max_new_tokens=1024,
    do_sample=True,
    top_p=0.95,
    temperature=0.7
)

response = processor.decode(outputs[0], skip_special_tokens=True)
print(response)

Chat Template Format

The model uses the following multimodal template format:

{{- bos_token }}
{%- for message in messages %}
    {%- if message['role'] == 'system' %}
        {{- '<start_of_turn>system\n' + message['content'] + '<end_of_turn>\n' }}
    {%- elif message['role'] == 'user' %}
        {{- '<start_of_turn>user\n' }}
        {%- if 'images' in message and message['images'] %}
            {%- for image in message['images'] %}
                {{- '<start_of_image>\n<end_of_image>\n' }}
            {%- endfor %}
        {%- endif %}
        {{- message['content'] + '<end_of_turn>\n' }}
    {%- elif message['role'] == 'assistant' %}
        {{- '<start_of_turn>model\n' + message['content'] + '<end_of_turn>\n' }}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt and messages[-1]['role'] != 'assistant' %}
    {{- '<start_of_turn>model\n' }}
{%- endif %}

Step-by-Step Reasoning Examples

LogicFlow-Gemma-3-27b-thinking demonstrates exceptional reasoning capabilities through detailed Chain-of-Thought (CoT) processes. Below are real examples showcasing the model's thinking methodology:

Example 1: Mathematical Comparison

Question: "9.11 and 9.9, which one is larger?"

The model demonstrates sophisticated numerical reasoning by:

• Converting decimals to fractional comparisons (11/100 vs 90/100)
• Using multiple verification methods (number line visualization, real-world applications)
• Calculating the precise difference (0.79) to confirm the result
• Providing comprehensive step-by-step analysis

Example 2: Letter Counting Task

Question: "How many r's are in the word strawberry?"

The model showcases systematic thinking through:

• Letter-by-letter breakdown of the word "strawberry"
• Multiple verification approaches (position counting, pattern grouping)
• Cross-checking results using different methodologies
• Clear documentation of the reasoning process

These examples demonstrate the model's ability to:

• Break down complex problems into manageable steps
• Self-verify results using multiple approaches
• Document reasoning chains for transparency
• Maintain accuracy while showing work

Activating Chain-of-Thought Reasoning

To get the best reasoning performance from LogicFlow-Gemma-3-27b-thinking, use prompts that encourage step-by-step thinking:

# Example prompt for mathematical reasoning
prompt = """Please solve this problem step by step, showing your thinking process:

Question: Compare 9.11 and 9.9. Which number is larger?

Think through this carefully and show your work."""

# Example prompt for logical reasoning  
prompt = """Let me work through this systematically:

Question: How many times does the letter 'r' appear in the word 'strawberry'?

Please show your step-by-step analysis."""

# For complex problems, you can explicitly request thinking
prompt = """Think step by step about this problem:

[Your complex question here]

Show your reasoning process before giving the final answer."""

Pro Tips for Best Results:

• Use phrases like "step by step", "think through this", "show your work"
• For math problems, request multiple verification methods
• Ask for reasoning before the final answer
• Use temperature settings around 0.7 for optimal reasoning creativity

Intended Use Cases

This multimodal model is particularly well-suited for:

Educational Applications

• Chain-of-Thought Tutoring: Demonstrates complete problem-solving processes with transparent reasoning steps
• Mathematical Education: Shows multiple verification methods for mathematical concepts (as seen in 9.11 vs 9.9 example)
• Critical Thinking Development: Models systematic analysis and self-verification techniques
• Visual Learning: Analyzing educational diagrams, charts, and mathematical illustrations
• Interactive Learning: Combining text and visual elements for comprehensive understanding

Mathematical & Scientific Analysis

• Chart Analysis: Interpreting graphs, statistical charts, and data visualizations
• Geometric Problem Solving: Analyzing geometric figures and spatial relationships
• Scientific Diagram Understanding: Processing scientific illustrations and technical drawings
• Formula Recognition: Understanding mathematical formulas in images

Professional Applications

• Document Analysis: Processing documents containing both text and visual elements
• Technical Documentation: Understanding technical manuals with diagrams
• Data Visualization: Analyzing and explaining complex charts and infographics
• Research Assistance: Combining textual research with visual data analysis

Advanced Reasoning Tasks

• Chain-of-Thought Problem Solving: Complex reasoning with detailed step-by-step analysis and self-verification
• Multi-Method Validation: Using multiple approaches to verify answers (numerical comparison, pattern analysis, etc.)
• Transparent Decision Making: Showing complete reasoning chains for critical analysis tasks
• Multimodal Problem Solving: Tackling problems that require both visual and textual understanding
• Visual Code Analysis: Understanding flowcharts, UML diagrams, and code structure visualizations
• Pattern Recognition: Identifying patterns in both visual and textual data

Limitations

Text Generation

• The model may occasionally generate incorrect mathematical calculations despite showing proper reasoning steps
• Performance on highly specialized domain knowledge outside of mathematics and logic may be limited
• As with all language models, it can sometimes produce hallucinated information

Vision Understanding

• Image Resolution: Images are resized to 896x896 pixels, which may lose important details in high-resolution images
• Image Quality: Poor quality, blurry, or low-contrast images may reduce accuracy
• Complex Visual Elements: Very dense charts or diagrams with small text may be challenging to interpret
• Image Formats: Only supports standard image formats (JPEG, PNG, WebP)

General Limitations

• The model should not be used for critical decision-making without human verification
• Multimodal reasoning combining complex visual and textual elements may sometimes produce inconsistent results
• Processing images increases computational requirements and inference time

Ethical Considerations

• This model should be used responsibly and outputs should be verified, especially for important decisions
• The model may reflect biases present in its training data
• Users should be aware that the model's reasoning, while often sound, is not infallible

Complete Training Configuration

For full reproducibility, here is the complete training configuration used:

bf16: true
cutoff_len: 2048
dataset: openo1_sft,open_thoughts,open_r1_math  # Three specialized reasoning datasets
dataset_dir: data
ddp_timeout: 180000000
do_train: true
enable_thinking: true
finetuning_type: lora
flash_attn: auto
freeze_multi_modal_projector: true
freeze_vision_tower: true
gradient_accumulation_steps: 8
image_max_pixels: 589824
image_min_pixels: 1024
include_num_input_tokens_seen: true
learning_rate: 5.0e-05
logging_steps: 5
lora_alpha: 16
lora_dropout: 0
lora_rank: 8
lora_target: all
lr_scheduler_type: cosine
max_grad_norm: 1.0
max_samples: 100000
model_name_or_path: google/gemma-3-27b-it
num_train_epochs: 5.0
optim: adamw_torch
output_dir: saves/Gemma-3-27B-Instruct/lora/train_2025-06-12-17-10-14
packing: false
per_device_train_batch_size: 2
plot_loss: true
preprocessing_num_workers: 16
report_to: none
save_steps: 100
stage: sft
template: gemma
trust_remote_code: true
video_max_pixels: 65536
video_min_pixels: 256
warmup_steps: 100

Technical Specifications

Core Framework

• Framework: Transformers 4.52.4
• PEFT Version: 0.15.2
• PyTorch Version: 2.7.0+cu126
• Training Framework: LLaMA-Factory with LoRA fine-tuning

Hardware Requirements

• Recommended GPU Memory: 32GB+ VRAM for multimodal inference
• Minimum GPU Memory: 24GB VRAM (text-only mode)
• CPU Memory: 64GB+ RAM recommended for optimal performance
• Quantization: Supports 4-bit and 8-bit quantization for reduced memory usage

Vision Specifications

• Vision Model: SIGLIP-based vision encoder
• Image Resolution: 896x896 pixels (normalized)
• Image Patch Size: 14x14 pixels
• Vision Hidden Size: 1,152
• Vision Layers: 27 layers
• Tokens per Image: 256 tokens
• Supported Image Formats: JPEG, PNG, WebP

Architecture Details

• Model Architecture: Gemma3ForConditionalGeneration
• Text Hidden Size: 5,376
• Vision Hidden Size: 1,152
• Attention Heads: 32 (text), 16 (vision)
• Hidden Layers: 62 (text), 27 (vision)
• Context Window: 131,072 tokens (including image tokens)

Citation

If you use this model in your research or applications, please cite:

@model{logicflow-gemma-3-27b-thinking,
  title={LogicFlow-Gemma-3-27b-thinking: A Fine-tuned Model for Enhanced Reasoning},
  author={[Xiangda Li]},
  year={2025},
  base_model={google/gemma-3-27b-it},
  url={https://huggingface.co/RekklesAI/LogicFlow-Gemma-3-27b-thinking}
}

Acknowledgments

• Based on Google's Gemma-3-27B-IT model
• Fine-tuned using LLaMA-Factory framework
• Training data from open-source reasoning and mathematics datasets

---

This model card was generated to provide comprehensive information about the LogicFlow-Gemma-3-27b-thinking model. Please refer to the original Gemma-3 model documentation for additional technical details about the base architecture.