Downloads · 30 days
16
13% of all-time downloads
Deepseek-R1-W4AFP8
novita/Deepseek-R1-W4AFP8
Deepseek-R1-W4AFP8 is a machine learning model from novita. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
- Model Architecture: DeepseekV3ForCausalLM - Input: Text - Output: Text - Model Optimizations: - Dense Weight quantization: FP8 - MOE Weight quantization: INT4 - Activation quantization: FP8 - Release Date: 25/10/202…
All-time downloads
127
Public
Parameters
349B
366 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors366 GB · 100%
Parameter types
How the weights are stored.
I8327B · 94%
Model card
From the Hugging Face model README
Deepseek-R1-W4AFP8
Model Overview
- Model Architecture: DeepseekV3ForCausalLM
- Input: Text
- Output: Text
- Model Optimizations:
- Dense Weight quantization: FP8
- MOE Weight quantization: INT4
- Activation quantization: FP8
- Release Date: 25/10/2025
- Version: 1.0
Quantized version of deepseek-ai/Deepseek-R1-W4AFP8
| Model | MMLU |
|---|---|
| novita/Deepseek-R1-W4AFP8 | 0.8705 |
Model Optimizations
These models were obtained by quantizing the weights and activations of DeepSeek models to mixed-precision data types (W4(int)A(FP)8 for MoE layers and FP8 for dense layers). This optimization reduces the number of bits per parameter 4/8, significantly reducing GPU memory requirements.
Use with SGLANG
This model can be deployed efficiently using the SGLANG backend with only H200x4, as shown in the example below.
python -m sglang.launch_server --model novita/Deepseek-R1-W4AFP8 --mem-fraction-static 0.85 --disable-shared-experts-fusion --tp-size 4
- Type
- deepseek_v3
- License
- apache-2.0
- Created
- Oct 24, 2025
- Updated
- Oct 27, 2025