Downloads · 30 days
38
1% of all-time downloads
DeepSeekV3
pandeyps/DeepSeekV3
DeepSeekV3 is a text generation model from pandeyps. Use it when you need the model to write or continue text. It is set up for pytorch. The card lists the license as mit.
Implementation of the DeepSeek-V3 architecture (8x2 MoE).
All-time downloads
3.8K
Public
Parameters
170M
679 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors679 MB · 100%
At a glance
- Task
- Text Generation
- Library
- pytorch
- License
- mit
- Model type
- deepseek_v3_mini
- Access
- Public
- Created
- Apr 13, 2026
- Updated
- Apr 13, 2026
- SHA
- f3ba1d2c
Try a prompt
Base models
Model card
From the Hugging Face model README
DeepSeekV3
Implementation of the DeepSeek-V3 architecture (8x2 MoE).
Architecture Details
- Architecture: Mixture of Experts (MoE) + Multi-Head Latent Attention (MLA)
- Parameters: ~196 million
- Layers: 6
- Hidden Dimension: 512
- Experts: 8 routed experts (Top-2) + 1 Shared Expert
- Attention: MLA (Multi-Head Latent Attention) with 8 heads and 64-dim Latent Space
- Position Embeddings: Sinusoidal
Training
- Dataset: TinyStories
- Platform: Kaggle (2x T4 GPUs)
- Optimizer: AdamW
How to use
This model uses a custom architecture implementation. To load it, you can use the state_dict found in model.safetensors alongside a loader.
from safetensors.torch import load_file
weights = load_file("model.safetensors")
Dataset
Trained on the TinyStories dataset.
- Task
- Text Generation
- Library
- pytorch
- Type
- deepseek_v3_mini
- License
- mit
- Languages
- en
- Created
- Apr 13, 2026
- Updated
- Apr 13, 2026