Downloads · 30 days
175
17% of all-time downloads
DeepSeek-V4-Pro-NVFP4-FP8
RedHatAI/DeepSeek-V4-Pro-NVFP4-FP8
DeepSeek-V4-Pro-NVFP4-FP8 is a text generation model from RedHatAI. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This is a quantized version of deepseek-ai/DeepSeek-V4-Pro with MoE layers quantized to NVFP4 and attention layers quantized to FP8 block
All-time downloads
1.1K
Public
Parameters
894B
2.7 TB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors911 GB · 100%
Parameter types
How the weights are stored.
U8776B · 87%
Try a prompt
Base models
Model card
From the Hugging Face model README
RedHatAI/DeepSeek-V4-Pro-NVFP4-FP8
This is a quantized version of deepseek-ai/DeepSeek-V4-Pro with MoE layers quantized to NVFP4 and attention layers quantized to FP8 block
Usage
This model is intended for deployment with vLLM and requires the following branch: https://github.com/vllm-project/vllm/pull/41276. You can serve the model using
vllm serve RedHatAI/DeepSeek-V4-Pro-NVFP4-FP8-BLOCK --tensor_parallel_size 8 --kv_cache_dtype=fp8
Creation Process
This model was created using LLM Compressor. The example script can be found in examples/quantizing_moe/deepseek_v4_pro_example.py [DSV4] DeepSeekV4 Pro. Quantizing the model with data parallelism and 6xA100 takes about 3 hours.
Evaluation
| Benchmark | deepseek-ai/DeepSeek-V4-Pro-Base | deepseek-ai/DeepSeek-V4-Pro | RedHatAI/DeepSeek-V4-Pro-NVFP4-FP8 |
|---|---|---|---|
| GPQA | 90.1 | 0.93 (380/792 samples) | |
| GSM8K | 91.1 | 92.6 | 91.0 |
- Task
- Text Generation
- Library
- transformers
- Type
- deepseek_v4
- License
- mit
- Created
- Jun 29, 2026
- Updated
- Jul 24, 2026