grug-27b-oQ6e-fp16

grug-27b-oQ6E-fp16

grug-27b-oQ6E-fp16 is a highly token-efficient reasoning model built on top of ProCreations/grug-27b, quantized with oMLX oQ6E (6-bit exponent-aware) using oMLX version 0.5.7, with a float16 (fp16) base dtype optimized for Apple Silicon performance.

Model Description

grug-27b-oQ6E-fp16 is a quantized variant of the grug-27b model, which itself is a LoRA fine-tune of Qwen/Qwen3.6-27B. The model employs the distinctive "grug" thinking style — a token-efficient reasoning approach that dramatically reduces token consumption while maintaining strong benchmark performance.

Key Features

• Token-efficient reasoning: Up to 198x fewer tokens than the base model on reasoning tasks
• oQ6E quantization: 6-bit exponent-aware quantization via oMLX for minimal quality loss
• fp16 base dtype: Float16 base dtype provides ~20% faster prefill on M1/M2 Apple Silicon
• LoRA fine-tuned: r=32 fine-tune on all text-stack linear layers, merged to bf16
• Zero repetition loops: Stress-tested with no repetition loop failures

Architecture

Quantization Details

This model is quantized using the oMLX library (version 0.5.7) with the oQ6E format — a 6-bit exponent-aware quantization scheme.

Note: The fp16 base dtype is selected for ~20% faster prefill on M1/M2/M3/M4 Apple Silicon. The output model name appends -fp16 to distinguish it from the bf16 variant.

Training Details

Fine-tuning

• Base Model: Qwen/Qwen3.6-27B
• LoRA Rank (r): 32
• Target Layers: All text-stack linear layers
• Merge Strategy: LoRA weights merged into bf16 base model

Training Data

• grug-think-v3-10k — agent trajectories (think-only loss)
• Fresh gpt-5.5 dataset — high-quality general data (full loss)

Loss Strategy

• Trajectory data: Think-only loss on reasoning trajectories
• Fresh data: Full token-level loss on high-quality general data

Performance Benchmarks

Benchmarks reported under v2.1 evaluation:

Token Efficiency

Hardware Requirements

Note: This model is designed for Apple Silicon (MLX) deployment. The fp16 dtype provides optimal performance on M1/M2 series chips; Pro/Max variants benefit from larger memory for longer contexts.

Usage

Python (MLX-LM)

from mlx_lm import load, generate

model, tokenizer = load("your-username/grug-27b-oQ6E-fp16")
response = generate(
    model, tokenizer,
    prompt="Your prompt here",
    max_tokens=1024,
    temp=0.6,
    top_p=0.95
)
print(response)

CLI

mlx_lm.run   --model your-username/grug-27b-oQ6E-fp16   --max-tokens 4096   --temp 0.6   --top-p 0.95

Recommended Inference Parameters

License

This model is derived from ProCreations/grug-27b, which is licensed under the Apache License 2.0.

The quantization and fp16 conversion are performed using oMLX (version 0.5.7).

See the LICENSE file for full terms.

Citation

@misc{grug27b,
  title={grug-27b},
  author={ProCreations},
  year={2025},
  url={https://huggingface.co/ProCreations/grug-27b}
}

@misc{qwen3,
  title={Qwen3.6-27B},
  author={Qwen Team},
  year={2025},
  url={https://huggingface.co/Qwen/Qwen3.6-27B}
}

Acknowledgements

• Qwen Team for the Qwen3.6-27B base model
• ProCreations for the grug-27b fine-tune
• oMLX for the oQ6E quantization format
• MLX for the Apple Silicon inference framework