Qwen3.8-2.4T-A95B-NVFP4-REAP-50

Qwen3.8-2.4T-A95B-NVFP4-REAP-50

Model Overview

• Model Architecture: Qwen3_5MoeForCausalLM
  • Input: Text
  • Output: Text
• Model Optimizations:
  • Weight quantization: FP4
  • Activation quantization: FP4
  • Expert pruning: 50% REAP Pruning
• Release Date: 2026-08-19
• Version: 1.0
• Model Developers: RedHatAI

This model is a quantized version of Qwen/Qwen3.8-2.4T-A95B. It was evaluated on several tasks to assess its quality in comparison to the unquantized model.

Model Optimizations

This model was obtained by applying mixed-precision quantization to Qwen/Qwen3.8-2.4T-A95B with 50% REAP Pruning of the experts: the MoE expert linear layers use FP4 (NVFP4) weights and activations, ready for inference with vLLM. Attention and non-quantized layers keep their original precision.

This optimization reduces the number of bits per parameter in the quantized MoE layers from 16 to 4, and applies 50% REAP Pruning to the experts, reducing the disk size and GPU memory requirements of those layers by approximately 75% plus the sparsity gain.

Only the weights and activations of the linear operators in the MoE experts are quantized using LLM Compressor.

Deployment

vLLM Serving

vllm serve RedHatAI/Qwen3.8-2.4T-A95B-NVFP4-REAP-50 \
    --tensor-parallel-size 8 \
    --enable-expert-parallel \
    --reasoning-parser qwen3

Creation

This model was created by applying LLM Compressor with the NVFP4 scheme and 50% REAP Pruning of the experts, exported in compressed-tensors

Accuracy

This model was evaluated on GPQA Diamond and DeepSWE (v1.1), served with vLLM (OpenAI-compatible API). Recovery is computed against the unquantized model.

<table> <thead> <tr> <th>Category</th> <th>Benchmark</th> <th>Qwen/Qwen3.8-2.4T-A95B</th> <th>RedHatAI/Qwen3.8-2.4T-A95B-NVFP4-REAP-50</th> <th>Recovery</th> </tr> </thead> <tbody> <tr> <td><b>Reasoning</b></td> <td>GPQA Diamond</td> <td>92.6</td> <td>90.7</td> <td>97.95%</td> </tr> <tr> <td><b>Coding</b></td> <td>DeepSWE (v1.1)</td> <td>56.6</td> <td>50.4</td> <td>89.05%</td> </tr> </tbody> </table>