Downloads · 30 days
12
12% of all-time downloads
md3p-int8
lewi/md3p-int8
md3p-int8 is a machine learning model from lewi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
An INT8 quantized version of Moondream3, offering a balance between model quality and size for MLX deployment.
All-time downloads
98
Public
Parameters
3.5B
11 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors11 GB · 100%
Parameter types
How the weights are stored.
U322B · 58%
Base models
Model card
From the Hugging Face model README
MD3P-Int8 - INT8 Quantized Moondream3 for MLX
An INT8 quantized version of Moondream3, offering a balance between model quality and size for MLX deployment.
Model Details
| Component | Original (BF16) | This Model |
|---|---|---|
| MoE Experts (layers 4-23) | BF16 | int8 |
| Vision Encoder | BF16 | BF16 (preserved) |
| Text Attention | BF16 | int8 |
| Text MLP (layers 0-3) | BF16 | int8 |
| Embeddings | BF16 | BF16 (preserved) |
| Total Size | ~12 GB | ~10 GB |
Quantization Details
- Method: Affine quantization (bits=8, group_size=64)
- Target: Text model layers (attention, MLP, MoE experts)
- Preserved: Vision encoder and embeddings at BF16 for quality
Comparison with INT4 Variants
| Model | Size | Quality | Use Case |
|---|---|---|---|
| md3p-int8 (this) | 10 GB | Higher | Desktop/Server MLX |
| md3p-int4 | 6.48 GB | Medium | Memory-constrained |
| md3p-int4-smol | 5.43 GB | Lower | iOS (~6GB limit) |
Usage
This model is designed for use with MLX-based Moondream implementations.
# Example with mlx-lm or similar
from mlx_lm import load, generate
model, tokenizer = load("lewi/md3p-int8")
Source & License
- Original Model: moondream/moondream3-preview
- License: Apache 2.0 (same as original)
Acknowledgments
Thanks to the Moondream team for the original model and Apache 2.0 license.
- Library
- mlx
- Type
- moondream3
- License
- apache-2.0
- Created
- Jan 16, 2026
- Updated
- Jan 16, 2026