Downloads · 30 days
140
18% of all-time downloads
DeepSeek-R1-Distill-Llama-8B-MNN
darkmaniac7/DeepSeek-R1-Distill-Llama-8B-MNN
DeepSeek-R1-Distill-Llama-8B-MNN is a text generation model from darkmaniac7. Use it when you need the model to write or continue text. The card lists the license as mit.
Published by TokForge for on-device inference.
All-time downloads
790
Public
Repo size
5.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.weight4.2 GB · 80%
At a glance
- Task
- Text Generation
- License
- mit
- Access
- Public
- Created
- Apr 2, 2026
- Updated
- Oct 12, 2026
- SHA
- 52939777
Try a prompt
Base models
Model card
From the Hugging Face model README
Published by TokForge for on-device inference.
TokForge
- Website: https://tokforge.ai
- Discord: https://discord.gg/Acv3CBtfVm
- Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
- iOS TestFlight: https://testflight.apple.com/join/jnufjzRr
Runs on-device in the TokForge app.
DeepSeek-R1-Distill-Llama-8B-MNN
Pre-converted DeepSeek R1 Distill Llama 8B in MNN format for on-device inference with TokForge.
Original model by DeepSeek, converted to MNN Q4 for mobile deployment.
Model Details
| Architecture | Llama 3.1 (standard attention, 32 layers, GQA, distilled from DeepSeek-R1) |
| Parameters | 8B (4-bit quantized) |
| Format | MNN (Alibaba Mobile Neural Network) |
| Quantization | W4A16 (4-bit weights, block size 128) |
| Vocab | 128,256 tokens |
| Source | deepseek-ai/DeepSeek-R1-Distill-Llama-8B |
Description
DeepSeek's R1 reasoning capability distilled into a Llama 3.1 8B body. Brings chain-of-thought reasoning to mobile devices. Shows its thinking process step-by-step, making it excellent for math, logic puzzles, coding, and complex analysis.
Files
| File | Description |
|---|---|
llm.mnn | Model computation graph |
llm.mnn.weight | Quantized weight data (Q4, block=128) |
llm_config.json | Model config with Jinja chat template |
tokenizer.txt | Tokenizer vocabulary |
config.json | MNN runtime config |
Usage with TokForge
This model is optimized for TokForge, a free Android app for private, on-device LLM inference.
- Download TokForge from Google Play
- Open the app → Models → Download this model
- Start chatting: runs 100% locally, no internet required
Recommended Settings
| Setting | Value |
|---|---|
| Backend | OpenCL (Qualcomm) / Vulkan (MediaTek) / CPU (fallback) |
| Precision | Low |
| Threads | 4 |
| Thinking | Off (or On for thinking-capable models) |
Credits
- DeepSeek-R1-Distill-Llama-8B by DeepSeek (Hugging Face). Used under the MIT license.
- Llama 3.1 8B by Meta (Hugging Face). Used under the Llama 3.1 Community License. The distilled weights derive from it.
- Built with Llama. Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. The license text and this notice are in the
LICENSEandNOTICEfiles in this repo. - Converted with llmexport from MNN by Alibaba (Apache-2.0).
Limitations
- Intended for TokForge / MNN on-device inference on Android
- This is a runtime bundle, not a standard Transformers training checkpoint
- Quantization (Q4) may slightly reduce quality compared to the full-precision original
Community
- Website: tokforge.ai
- Discord: Join our Discord
Export Details
Converted using MNN's llmexport pipeline:
python llmexport.py --path deepseek-ai/DeepSeek-R1-Distill-Llama-8B --export mnn --quant_bit 4 --quant_block 128
- Task
- Text Generation
- License
- mit
- Languages
- en
- Created
- Apr 2, 2026
- Updated
- Oct 12, 2026