Downloads · 30 days
133
3% of all-time downloads
DeepSeek-V4-Flash-TQ-Q5.6-MLX
lemuralabs/DeepSeek-V4-Flash-TQ-Q5.6-MLX
DeepSeek-V4-Flash-TQ-Q5.6-MLX is a text generation model from lemuralabs. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as mit.
<p align="center" <img src="logo.png" alt="Lemura Labs" width="110"/ </p
All-time downloads
4.7K
Public
Parameters
56B
203 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors203 GB · 100%
Parameter types
How the weights are stored.
U3255.8B · 100%
Try a prompt
Base models
Model card
From the Hugging Face model README
<p align="center">
<img src="logo.png" alt="Lemura Labs" width="110"/>
</p>
DeepSeek-V4-Flash-TQ-Q5.6-MLX
lemuralabs/DeepSeek-V4-Flash-TQ-Q5.6-MLX is an Apple-Silicon MLX TurboQuant/JANGTQ quantization of deepseek-ai/DeepSeek-V4-Flash.
No fine-tuning, distillation, or retraining was applied. The official mixed FP4/FP8 source weights were converted locally, the MTP head was dropped because it is not used for normal decode, and router/mHC/control tensors were preserved rather than aggressively quantized.
Model Details
| Property | Value |
|---|---|
| Base model | deepseek-ai/DeepSeek-V4-Flash |
| Architecture | DeepSeek-V4 Flash MoE, 284B total / 13B active, 1M context |
| Local profile | JANGTQ-Q5.6 |
| Bundle size | 202.91 GB |
| Layout | Pre-stacked MLX switch_mlp layout |
| MTP head | Dropped |
| Validation | Safetensors header/index validation, metadata validation |
Required Sidecar
This is a JANGTQ/TurboQuant bundle and requires jangtq_runtime.safetensors from this repository. The sidecar stores the deterministic codebooks and Hadamard rotation signs used to decode the .tq_packed expert weights. If it is missing, re-download the full repository or fetch that file explicitly:
hf download lemuralabs/DeepSeek-V4-Flash-TQ-Q5.6-MLX jangtq_runtime.safetensors --local-dir <your-model-dir>
Quantization Recipe
| Tensor class | Codec | Bits / handling |
|---|---|---|
| Routed experts | TurboQuant MXTQ | 77 routed layer/projection groups at 4-bit MXTQ and 52 at 8-bit MXTQ |
| Routed effective bits | MXTQ | 5.6124 bits |
| Attention, shared experts, compressor, indexer, embed, lm head | MLX affine | 8-bit, group size 32 |
| Norms, router, mHC, sinks, integer routing tables | passthrough | source precision preserved |
The fractional target is implemented as a power-of-two lane mix because the current JANGTQ vectorized packer is stable on 2/4/8-bit lanes for DeepSeek-V4 expert dimensions.
Use
Install the JANG loader/runtime and MLX LM:
pip install mlx-lm jang-tools
Example:
from jang_tools.load_jangtq import load_jangtq_model
from mlx_lm import generate
model, tokenizer = load_jangtq_model("lemuralabs/DeepSeek-V4-Flash-TQ-Q5.6-MLX")
prompt = "Write a short note about MLX quantization."
text = generate(model, tokenizer, prompt=prompt, verbose=True)
print(text)
Files
model-*.safetensors: pre-stacked JANGTQ/MLX shardsmodel.safetensors.index.json: shard indexjangtq_runtime.safetensors: required TurboQuant runtime sidecarconfig.json,jang_config.json: MLX/JANGTQ metadataencoding/: upstream DeepSeek-V4 prompt encoding reference
Notes
This upload follows the same broad model-card shape as the public OsaurusAI/JANGQ DeepSeek-V4-Flash JANGTQ uploads: a sidecar warning, an explicit recipe table, and minimal reproducible loading instructions.
License
MIT, following the upstream DeepSeek-V4-Flash release.
- Task
- Text Generation
- Library
- mlx
- Type
- deepseek_v4
- License
- mit
- Languages
- en, zh
- Created
- May 17, 2026
- Updated
- Aug 5, 2026