Downloads · 30 days
70
8% of all-time downloads
MoAI-gemma-4-31B-it-mxfp4
BCCard/MoAI-gemma-4-31B-it-mxfp4
MoAI-gemma-4-31B-it-mxfp4 is a machine learning model from BCCard. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for vllm. The card lists the license as apache-2.0.
google/gemma-4-31B-it quantized to MXFP4 (W4A16) using GPTQ with special techniques that preserve Korean and English performance simultaneously.
All-time downloads
909
Public
Parameters
17.5B
39.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors19.5 GB · 100%
Parameter types
How the weights are stored.
U815.6B · 89%
Base models
Model card
From the Hugging Face model README
Gemma 4 31B IT — MXFP4 Quantized
google/gemma-4-31B-it quantized to MXFP4 (W4A16) using GPTQ with special techniques that preserve Korean and English performance simultaneously.
Model Details
| Property | Value |
|---|---|
| Base model | google/gemma-4-31B-it (BF16, 62.5 GB) |
| Quantization format | MXFP4 W4A16 (compressed-tensors) |
| Disk size | 19.5 GB (69% reduction from BF16) |
| VRAM (model weights) | ~19 GB |
| Calibration data | BCCard on-policy 600k |
Serving
vLLM (recommended)
# Standard environment (CUDA 12.9+):
vllm serve gemma-4-31B-it-MXFP4 \
--port 8000 \
--max-model-len 32768 \
--gpu-memory-utilization 0.90
# Blackwell with CUDA < 12.9:
VLLM_USE_FLASHINFER_SAMPLER=0 \
vllm serve gemma-4-31B-it-MXFP4 \
--port 8000 \
--linear-backend marlin \
--max-model-len 32768 \
--gpu-memory-utilization 0.90
Blackwell Compatibility
On Blackwell GPUs (SM 12.x) with containers running CUDA < 12.9, FlashInfer JIT compilation fails. The following workarounds are required:
| Setting | Purpose |
|---|---|
VLLM_USE_FLASHINFER_SAMPLER=0 | Disables FlashInfer sampler (uses native PyTorch) |
--linear-backend marlin | Uses pre-compiled MXFP4 GEMM kernels (no JIT) |
Component matrix after workarounds:
| Component | Backend | JIT Required |
|---|---|---|
| MXFP4 GEMM | Marlin (pre-compiled) | No |
| Attention | TRITON_ATTN | No (Triton JIT, not nvcc) |
| Sampling | Native PyTorch | No |
Evaluation
Quality Benchmarks
| Task | MXFP4 (this model) | BF16 baseline | Recovery |
|---|---|---|---|
| HellaSwag (English) | 89.5% | 91.0% | 98.4% |
| KoBEST-HellaSwag | 73.0% | 76.5% | 95.4% |
| KoBEST-COPA | 99.0% | 99.0% | 100.0% |
| KoBEST-BoolQ | 95.0% | 96.5% | 98.4% |
| KoBEST-SentiNeg | 99.0% | 99.0% | 100.0% |
| KoBEST-WiC | 98.5% | 99.0% | 99.5% |
| KMMLU (5 subjects) | 76.0% | 78.0% | 97.4% |
| BC-Finance QA (char-F1) | 0.387 | 0.386 | 100.3% |
| Average recovery | 98.7% | ||
| Korean Generation | 476 tok, rep 0.99, Hangul 59.2% | 482 tok, rep 0.99, Hangul 57.9% | — |
| English Generation | 424 tok, rep 0.99 | 421 tok, rep 1.00 | — |
Korean Knowledge (KMMLU) — Reference Points
Published scores from other Korean(-capable) models, for orientation. These are NOT apples-to-apples: vendors use the full 45-subject KMMLU (or the harder KMMLU-Redux subset) with their own prompting/shot settings, while this model's score is a 5-subject, 200-sample, 0-shot chat-MCQ sample from the harness above.
| Model | Params | Disk | Score | Benchmark (setting/source) |
|---|---|---|---|---|
| A.X 4.0 (SKT) | 72B | ~144 GB | 78.3 | KMMLU, vendor |
| Gemma 4 31B IT (BF16 baseline) | 31B | 62.5 GB | 78.0 | KMMLU 5-subject sample, 0-shot chat MCQ (this harness) |
| This model (Gemma 4 31B MXFP4, 4-bit) | 31B | 19.5 GB | 76.0 | KMMLU 5-subject sample, 0-shot chat MCQ (this harness) |
| EXAONE 4.0 (LG) | 32B | ~64 GB | 75.2 | KMMLU, as reported in the HyperCLOVA X 32B Think report |
| GPT-4o (OpenAI) | — | — (API) | 72.5 | KMMLU, as reported by SKT A.X |
| HyperCLOVA X 32B Think (NAVER) | 32B | ~64 GB | 71.3 | KMMLU, vendor report |
| A.X 3.1 (SKT) | 34B | ~68 GB | 69.2 | KMMLU, vendor |
| A.X 3.1 Light (SKT) | 7B | ~14 GB | 61.7 | KMMLU, vendor |
| Gemma 3 27B IT (Google) | 27B | ~54 GB | 54.0 | KMMLU-Redux, KMMLU-Redux/Pro paper |
Disk sizes for third-party models are approximate BF16 weight footprints (params × 2 bytes); this repo's model and its BF16 baseline are measured on-disk values.
Directional takeaway: on this 5-subject sample the BF16 gemma-4-31B-it baseline (78.0) sits on par with A.X 4.0 72B (78.3), and the 4-bit MXFP4 model retains that tier — landing above EXAONE 4.0 32B and GPT-4o — at a 19.5 GB footprint. Against its own BF16 baseline under the identical harness, the model averages 98.7% recovery across all scored tasks.
Note: Perplexity cannot be measured with MXFP4 + Marlin backend due to NaN in
prompt_logprobs. Text generation and classification tasks work normally.
Standard Benchmarks (lm-eval-harness)
| Benchmark | BF16 Baseline | RedHatAI FP8 | Recovery% | This Model (MXFP4) |
|---|---|---|---|---|
| GSM8K | 96.11 | 96.22 | 100.1% | TBD |
| MMLU-Pro | 87.05 | 87.07 | 100.0% | TBD |
| IFEval | 94.58 | 94.89 | 100.3% | TBD |
RedHatAI FP8 reference: RedHatAI/gemma-4-31B-it-FP8-dynamic
Compression Comparison
| Format | Disk Size | vs BF16 | Notes |
|---|---|---|---|
| BF16 (original) | 62.5 GB | 100% | google/gemma-4-31B-it |
| FP8-dynamic | 33.3 GB | 53% | RedHatAI/gemma-4-31B-it-FP8-Dynamic — 89.6% FP8_E4M3 + 6.1% BF16 |
| This Model (MXFP4) | 19.5 GB | 31% | W4A16 GPTQ + MHR + PSA |
Citation
@misc{gemma4-mxfp4-mhr-psa,
title={Gemma 4 31B IT MXFP4 with Multilingual Hessian Regularization and PSA-GPTQ},
author={Taeyoung Lee},
year={2026},
url={https://huggingface.co/BCCard/MoAI-gemma-4-31B-it-mxfp4}
}
License
This model is licensed under the Apache 2.0.
- Library
- vllm
- Type
- gemma4
- License
- apache-2.0
- Languages
- ko, en
- Created
- Jul 11, 2026
- Updated
- Jul 12, 2026