DeepSeek-V4-Flash-4Expert

What is DeepSeek-V4-Flash-4Expert?

DeepSeek-V4-Flash is a 284B-parameter Mixture-of-Experts (MoE) language model with 13B activated parameters, supporting a context length of one million tokens. The original model uses tok=6 by default.

cloudyu/DeepSeek-V4-Flash-4Expert is the same model with the number of activated experts per token reduced from 6 → 4, while keeping all other weights identical. This change:

• Reduces inference compute by ~33% (fewer active experts per forward pass)
• Improves generation throughput by ~8–11%
• Maintains or improves accuracy on both code generation and knowledge benchmarks
• Uses the same FP4 + FP8 mixed precision format as the original

Key Changes from Original

The tid2eid (expert routing) weight tensors have been reshaped from [vocab_size, 6] to [vocab_size, 4] — only the first 4 columns are retained, matching the original training distribution order. No additional training or fine-tuning was performed; this is purely an inference-time configuration change.

Independent Evaluation Results

We evaluated the model against the original top_k=6 configuration on two benchmarks: HumanEval (code generation) and MMLU-Pro (multi-domain knowledge).

HumanEval (Pass@1)

##eval details

• Identical accuracy on code generation
• 12.7% faster generation

eval details

MMLU-Pro (Accuracy)

##eval details

• +3.69 percentage points higher accuracy
• 8.1% faster generation

eval details

Category Breakdown (MMLU-Pro)

STEM and business categories (math, engineering, business, chemistry, physics, computer science) show significant improvements with top_k=4, while humanities and life sciences show modest regression.

Summary

• Top_k=4 wins in all practical metrics: higher or equal accuracy, faster inference, lower memory bandwidth usage
• The improvement is particularly pronounced on math, engineering, and business reasoning tasks
• The original top_k=6 configuration provides marginal benefits only in humanities/life sciences categories
• For production deployment, top_k=4 is the recommended configuration

Full evaluation reports, scripts, and raw results are available in the eval/ directory of this repository.

Model Downloads

Chat Template

This release does not include a Jinja-format chat template. Instead, we provide a dedicated encoding folder with Python scripts and test cases demonstrating how to encode messages in OpenAI-compatible format into input strings for the model, and how to parse the model's text output. Please refer to the encoding folder for full documentation.

A brief example:

from encoding_dsv4 import encode_messages, parse_message_from_completion_text

messages = [
    {"role": "user", "content": "hello"},
    {"role": "assistant", "content": "Hello! I am DeepSeek.", "reasoning_content": "thinking..."},
    {"role": "user", "content": "1+1=?"}
]

# messages -> string
prompt = encode_messages(messages, thinking_mode="thinking")

# string -> tokens
import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("cloudyu/DeepSeek-V4-Flash-4E")
tokens = tokenizer.encode(prompt)

How to Run Locally

Please refer to the inference folder for detailed instructions on running DeepSeek-V4 locally, including model weight conversion and interactive chat demos.

For local deployment, we recommend setting the sampling parameters to temperature = 1.0, top_p = 1.0. For the Think Max reasoning mode, we recommend setting the context window to at least 384K tokens.

License

This repository and the model weights are licensed under the MIT License.

Contact

If you have any questions, please raise an issue or contact cloudyu on HuggingFace.

---

---

DeepSeek-V4-Flash-4E 中文说明

基于 DeepSeek-V4-Flash 的改进变体，将 top k 从 6 改为 4，实现最优推理效率。

HuggingFace: cloudyu/DeepSeek-V4-Flash-4E

概述

DeepSeek-V4-Flash 是一个 284B 参数的混合专家（MoE）语言模型，激活 13B 参数，支持百万 token 上下文长度。原始模型默认使用 num_experts_per_tok=6。

DeepSeek-V4-Flash-4E 将每 token 激活专家数从 6 减少为 4，保持所有权重不变。这一改动带来：

• 推理计算量减少约 33%（更少的激活专家）
• 生成吞吐量提升约 8–11%
• 准确率保持不变甚至更高
• 保持原有的 FP4 + FP8 混合精度 格式

与原始模型的关键区别

tid2eid（专家路由）权重张量已从 [vocab_size, 6] 重塑为 [vocab_size, 4]——仅保留前 4 列，与原始训练分布顺序一致。未进行任何额外训练或微调，纯属推理时配置调整。

独立评测结果

我们在 HumanEval（代码生成）和 MMLU-Pro（多领域知识问答）两个基准上进行了对比评测。

HumanEval (Pass@1)

• 代码生成准确率完全相同
• 速度快 12.7%

MMLU-Pro (Accuracy)

• 准确率高出 3.69 个百分点
• 速度快 8.1%

MMLU-Pro 分类别对比

STEM 和商科类别（math、engineering、business、chemistry、physics、computer science）在使用 top_k=4 时提升显著，而人文和生命科学类别略有下降。

总结

• Top_k=4 在所有实用指标上胜出： 更高或相等的准确率、更快推理速度、更低显存带宽消耗
• 在数学、工程和商业推理任务上优势尤为突出
• 原始 top_k=6 仅在人文/生命科学类别上略有优势
• 对于生产部署，top_k=4 是推荐配置

完整的评测报告、脚本和原始数据位于本仓库的 eval/ 目录下。

模型下载

聊天模板

本仓库不包含 Jinja 格式的聊天模板。我们提供了专用的 encoding 文件夹，内含 Python 脚本和测试用例，演示如何将 OpenAI 兼容格式的消息编码为模型输入，以及如何解析模型输出。请参考 encoding 文件夹获取完整文档。

简单示例：

from encoding_dsv4 import encode_messages, parse_message_from_completion_text

messages = [
    {"role": "user", "content": "你好"},
    {"role": "assistant", "content": "你好！我是 DeepSeek。", "reasoning_content": "思考中..."},
    {"role": "user", "content": "1+1=?"}
]

# messages -> 字符串
prompt = encode_messages(messages, thinking_mode="thinking")

# 字符串 -> tokens
import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("cloudyu/DeepSeek-V4-Flash-4E")
tokens = tokenizer.encode(prompt)

本地运行

请参考 inference 文件夹获取本地运行 DeepSeek-V4 的详细说明，包括模型权重转换和交互式聊天演示。

本地部署时建议设置采样参数为 temperature = 1.0, top_p = 1.0。对于 Think Max 推理模式，建议将上下文窗口设置为至少 384K tokens。

许可协议

本仓库和模型权重采用 MIT 许可协议。

联系方式

如有任何问题，请在 HuggingFace 上提 issue 或联系 cloudyu。