Qwen3-Coder-Next-8bit-g128

Model Card for Qwen3-Coder-Next-8bit-g128

Quantized Qwen/Qwen3-Coder-Next using mlx-lm to 8-bit with group_size 128 for main weights and fine-grained group_size 64 for MoE weights, with the aim of maximum accuracy for 8-bit quantization.

Updated Evaluation Results (February 13, 2026)

Comprehensive evaluation results from thorough testing using mlx_lm.evaluate with mmlu_pro (200 questions per domain, num_shots=1, temp=1.0, top_p=0.95, top_k=40, seed=123):

Direct Comparison Summary (8-bit g64 vs g128)

Average Performance

• 8-bit g64: 87.0% average
• 8-bit g128 (this model): 87.0% average
• Difference: 0.0 percentage points (virtually identical)

Key Observations

• Both 8-bit models show nearly identical average performance (87.0%)
• Performance differences are domain-specific rather than model-specific
• 8-bit g64 performs better in Math, Engineering, and Physics
• 8-bit g128 (this model) performs better in Computer Science and Chemistry, with Computer Science being the most significant improvement (+2.5%)

Memory Usage Comparison

• 8-bit g64:

  • Peak memory usage: 95.374 GB (Math) to 100.740 GB (Engineering)
  • Average memory usage: ~98.6 GB

• 8-bit g128 (this model):

  • Peak memory usage: 97.702 GB (Computer Science) to 98.510 GB (Engineering)
  • Average memory usage: ~97.8 GB

Full Quantization Spectrum Comparison (Updated)

Original Evaluation Results

Testing with mlx_lm.evaluate using mmlu_pro with 50 Qs for each topic, comparing the 8-bit g128 quant with the 8-bit g64 quant:

Average Performance

• 8-bit g64: 84.4% average
• 8-bit g128: 88.0% average
• Improvement: +3.6 percentage points with group size 128 (main weights) with fine-grained 64 (MoE weights)

Key Benefits

• The g128 group_size applied to main weights (with fine-grained g64 for MoE weights) is also ~2GB smaller than the g64 version
• At the time of uploading, there were not any 8bit g128 quants of this model available on HF
• Also tried MXFP8 but found equal or lower accuracy than g64 for all topics (Engineering dropped to 0.76 and Chemistry dropped to 0.88 (-4% and -2% compared to g64))

Model Details

• Base Model: Qwen/Qwen3-Coder-Next
• Library: mlx-lm
• Quantization: 8-bit with group_size 128 for main weights and group_size 64 for MoE weights
• License: apache-2.0
• Pipeline Tag: text-generation

Usage

import mlx_lm
from mlx_lm.sample_utils import make_sampler

model_path = "petergilani/qwen3-coder-next-8bit-g128"
model, tokenizer = mlx_lm.load(model_path)

sampler = make_sampler(temp=1.0, top_p=0.95, top_k=40)

prompt = "Write a Python function to calculate the factorial of a number."
response = mlx_lm.generate(
    model, 
    tokenizer, 
    prompt=prompt,
    sampler=sampler,
    max_tokens=512
)
print(response)