grug-27b-oQ8e-fp16

grug-27b-oQ8E-fp16

A token-efficient, quantized reasoning model based on Qwen3.6-27B with the "grug" thinking style.

This is an oMLX-oQ8E quantized variant of ProCreations/grug-27b, converted to float16 for ~20% faster prefill on Apple Silicon (M1/M2/M3/M4). The base model is Qwen/Qwen3.6-27B (Apache 2.0).

Model Description

grug-27b is a LoRA r=32 fine-tune of Qwen3.6-27B, trained on agent trajectories with a "grug" thinking style — a token-efficient reasoning approach that uses 198× fewer tokens than the base model while maintaining strong performance. The LoRA weights are merged into the bf16 backbone, and this variant further quantizes to oQ8E (float16) for efficient MLX deployment.

Key Features

• Token efficiency: 8–40× fewer tokens than base model for equivalent outputs
• Zero repetition loops: Stress-tested with no degenerate looping behavior
• Fast prefill: float16 dtype delivers ~20% faster prefill on Apple Silicon
• Exponent-aware quantization: oQ8E preserves numerical precision where it matters most

Architecture

Quantization Details

Performance Benchmarks (v2.1)

Training Data & Method

• Fine-tuning: LoRA r=32 on all text-stack linear layers, merged to bf16 backbone
• Trajectory data: grug-think-v3-10k agent trajectories
• Supplementary data: Fresh GPT-5.5 generated high-quality samples
• Loss strategy: Think-only loss on trajectory data, full loss on fresh high-quality data

Hardware Requirements

Usage

Python (MLX-LM)

from mlx_lm import load, generate

model, tokenizer = load("your-username/grug-27b-oQ8E-fp16")
response = generate(
    model, tokenizer,
    prompt="Your prompt here",
    max_tokens=1024,
    temp=0.6,
    top_p=0.95
)
print(response)

CLI

mlx_lm.run   --model your-username/grug-27b-oQ8E-fp16   --max-tokens 4096   --temp 0.6   --top-p 0.95

Recommended Generation Parameters

License

This model is derived from ProCreations/grug-27b, which is licensed under the Apache License 2.0. The base model Qwen/Qwen3.6-27B is also Apache 2.0.

This quantized variant inherits the same Apache 2.0 license. See LICENSE for details.

Citation

@misc{grug27b,
  title={grug-27b: Token-Efficient Reasoning with the Grug Thinking Style},
  author={ProCreations},
  year={2025},
  url={https://huggingface.co/ProCreations/grug-27b}
}

Acknowledgements

• Base model: Qwen/Qwen3.6-27B by Qwen Team
• Original model: ProCreations/grug-27b
• Quantization: oMLX oQ8E format (v0.5.7)