gemma-4-31B-Opus-4.6-Reasoning-GGUF

Gemma-4-31B-Opus-4.6-Reasoning-GGUF

This is a quantized GGUF version of the fine-tuned and merged Gemma-4-31B_Opus-4.6-Reasoning-3300x model.

Introduction

The primary goal of this project was to leverage Gemma-4's native <|channel> architecture to enforce strict, logical step-by-step reasoning. By fusing the Gemma-4 foundation with the Opus reasoning dataset, the model acts as a deeply analytical agent.

This repository provides ultra-efficient .gguf archives. These files enable you to run exactly this reasoning model entirely on consumer-grade hardware (like a MacBook or modest GPU setup) without requiring large amounts of VRAM.

Benchmarks & Performance

While this is a 4-bit quantization, the model retains the logical density of the original BF16 merge.

• Original BF16 ARC-Challenge (acc_norm): 69.54%
• Original Base (Gemma-4-31B) ARC-Challenge (acc_norm): 69.88%

The structural output improvements remain consistent across quants, maintaining strict <|channel>thought integrity.

Reasoning Format: Gemma 4 Architecture

This model adheres strictly to the Gemma 4 multimodal and reasoning formats. It outputs internal reasoning within <|channel>thought bounds before delivering the final response.

Below are raw, unedited samples from this quant:

User: I need to fine-tune a 70B parameter model. I have exactly two 80GB A100 GPUs. Walk me through the math of whether I can do a full-parameter fine-tune in pure 16-bit, or if I am mathematically forced to use 4-bit quantization and LoRA. Show your exact VRAM calculations.

Model: <|channel>thought Let me work through this step by step:

1. Model Size Calculation
   • Model size = 70B parameters * 2 bytes (fp16) = 140,000,000,000 bytes
   • Convert to GB: 140,000,000,000 / 1,024^3 ≈ 128.19 GB ...
2. Memory Comparison
   • Required: 512.76 GB
   • Available: 160 GB
   • Shortfall: 512.76 - 160 = 352.76 GB

Conclusion

With two 80GB A100s, you cannot perform a full-parameter fine-tune in pure 16-bit precision due to insufficient VRAM. <channel|> Final Answer: You will need to use either:

1. 4-bit quantization (QLoRA) with LoRA adapters
2. ZeRO-3 offloading ...

Training Details (from original BF16 model)

• Base Model: google/gemma-4-31B
• Dataset: Crownelius/Opus-4.6-Reasoning-3300x
• Training Framework: Eschaton Engine (Cloudbjorn)
• Format: Merged (Base + LoRA)

Training Precision:

• Compute Dtype: bfloat16

LoRA Parameters (Auto-Scaled for 31B via Eschaton):

• r: 16
• lora_alpha: 32
• target_modules: all-linear

Hyperparameters:

• Optimizer: 8-bit Paged AdamW
• Effective Batch Size: 32 (Gradient Accumulation)
• Learning Rate: 2e-5
• LR Scheduler: Linear
• Epochs: 1
• Training Sequence Length: 2048
• Warmup Steps: 50
• Weight Decay: 0.01