Downloads · 30 days
22
16% of all-time downloads
Phi-2-MLX-4bit
Irfanuruchi/Phi-2-MLX-4bit
Phi-2-MLX-4bit is a text generation model from Irfanuruchi. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as mit.
This repository provides a 4-bit MLX-quantized version of Microsoft Phi-2, optimized for fast, low-memory local inference on Apple Silicon.
All-time downloads
134
Public
Parameters
2.8B
1.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.6 GB · 100%
Parameter types
How the weights are stored.
U322.8B · 100%
Try a prompt
Base models
Model card
From the Hugging Face model README
Phi-2 MLX 4-bit
This repository provides a 4-bit MLX-quantized version of Microsoft Phi-2, optimized for fast, low-memory local inference on Apple Silicon.
This variant prioritizes speed and minimal RAM usage, making it ideal for laptops and on-device experimentation.
Model Details
- Base model: microsoft/phi-2
- Architecture: Decoder-only Transformer
- License: MIT
- Quantization: MLX static quantization (≈4.5 bits per weight)
- Target hardware: Apple Silicon (M1 / M2 / M3)
Performance Characteristics
| Metric | Value |
|---|---|
| Disk size | ~1.5–1.7 GB |
| Peak RAM usage | ~1.6–1.8 GB |
| Inference speed | Fast |
| Instruction quality | Good |
Usage
mlx_lm.generate \
--model /path/to/Phi-2-MLX-4bit \
--prompt "Explain the FFT in simple terms." \
--max-tokens 120
Notes
- This is a quantized conversion, not a fine-tune.
- The 4-bit version is best for:
- faster inference
- lower memory usage
- interactive local testing
- For higher-quality reasoning and instruction-following, see the 5-bit variant.
License
This repository redistributes a quantized MLX conversion of Microsoft Phi-2.
- Original model license: MIT
- MLX conversion: MIT
See LICENSE for details.
- Task
- Text Generation
- Library
- mlx
- Type
- phi
- License
- mit
- Languages
- en
- Created
- Feb 7, 2026
- Updated
- Feb 7, 2026