Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled

Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled : GGUF

Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled is a reasoning-enhanced language model fine-tuned from the Qwen3-4B-Thinking-2507 base model using a high-density synthetic reasoning dataset distilled from Gemini 3.1 Pro.

The goal of this fine-tuning is to improve multi-step reasoning, structured problem decomposition, and analytical thinking while maintaining the strong language capabilities of the base Qwen model.

This model focuses on System-2 style reasoning, emphasizing logical derivations, verification steps, and structured explanations.

• Model name: khazarai/Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled
• Base model: Qwen3-4B-Thinking-2507

Available Model files:

• qwen3-4b-thinking-2507.BF16.gguf
• qwen3-4b-thinking-2507.Q8_0.gguf
• qwen3-4b-thinking-2507.Q4_K_M.gguf

These files enable deployment in local inference frameworks such as llama.cpp or Ollama.

Training Method

Method:

• QLoRA (4-bit quantization)
• LoRA adapters
• Training framework: Unsloth

Dataset

The model was trained on a synthetic reasoning dataset distilled from Gemini 3.1 Pro.

Dataset Overview

Total Token Volume: 5.6 million tokens The dataset prioritizes logical density and deep reasoning traces, rather than large token counts. The dataset represents a frontier approach in reasoning distillation, where a stronger reasoning model generates structured problem-solving examples that are then used to train a smaller model.

Dataset generation pipeline:

• Question generation
• Multi-step reasoning derivation
• Answer verification

Curated using an agentic workflow involving:

• Gemini 3 Flash
• Gemini 3.1 Pro

Domain Coverage

The dataset spans 60+ expert-level domains, focusing on complex reasoning tasks.

• Physics
• Mathematics
• Computer Science
• Biology & Medicine
• Strategic Reasoning
• Advanced Benchmarks

The dataset includes problems inspired by challenging benchmarks such as:

• ARC-AGI
• GPQA Diamond
• MMLU-Pro
• HLE Benchmark

Intended Use

This model is designed for:

• Complex reasoning tasks
• Scientific explanations
• Advanced mathematics and logic
• Programming and computer science questions
• Analytical problem solving

Typical use cases include:

• research assistance
• algorithm reasoning
• technical education
• complex question answering

Limitations

• Despite improved reasoning ability, the model still inherits limitations from its base architecture:
• Hallucinations may still occur in specialized domains.
• Synthetic reasoning datasets may introduce stylistic biases.
• Performance may degrade on tasks requiring extremely long context or domain-specific expertise.

Users should verify outputs when used in critical scientific or technical environments.