Downloads · 30 days
10
9% of all-time downloads
DeepSeek-V2-Lite-MoNE-48-zyda2-100
MoNE-Pruning/DeepSeek-V2-Lite-MoNE-48-zyda2-100
DeepSeek-V2-Lite-MoNE-48-zyda2-100 is a text generation model from MoNE-Pruning. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This repository contains a structured pruned variant of DeepSeek-V2-Lite using the MoNE (Mixture-of-Novice Experts) framework proposed in our paper.
All-time downloads
109
Public
Parameters
12.1B
24.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors24.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- deepseek_v2_compressed
- Access
- Public
- Created
- Feb 28, 2026
- Updated
- Feb 28, 2026
- SHA
- 3022f8ee
Try a prompt
Base models
Model card
From the Hugging Face model README
DeepSeek-V2-Lite-MoNE-48-zyda2-100
This repository contains a structured pruned variant of DeepSeek-V2-Lite using the MoNE (Mixture-of-Novice Experts) framework proposed in our paper.
*## Model Overview
- Base Model: DeepSeek-V2-Lite
- Method: MoNE structured expert pruning
- Remaining Experts: 48
- Calibration Set: zyda2-100
- Architecture: Mixture-of-Experts (MoE)
- Framework: Transformers-compatible
This checkpoint replaces redundant experts with lightweight novice experts via structured pruning, aiming to reduce compute while preserving performance.
Paper
Title: MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoE
Authors: Geng Zhang, Yuxuan Han, Yuxuan Lou, Yiqi Zhang, Wangbo Zhao, Yang You
arXiv: arXiv:2507.00390
- Task
- Text Generation
- Library
- transformers
- Type
- deepseek_v2_compressed
- License
- mit
- Created
- Feb 28, 2026
- Updated
- Feb 28, 2026