Downloads · 30 days
26
7% of all-time downloads
MediPhi-Instruct-mlx-4bit
bisonnetworking/MediPhi-Instruct-mlx-4bit
MediPhi-Instruct-mlx-4bit is a text generation model from bisonnetworking. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as mit.
This repository contains an MLX-format 4-bit quantized version of microsoft/MediPhi-Instruct, converted using mlx-lm for efficient on-device inference on Apple silicon.
All-time downloads
358
Public
Parameters
3.8B
2.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.1 GB · 100%
Parameter types
How the weights are stored.
U323.8B · 100%
Try a prompt
Base models
Model card
From the Hugging Face model README
MediPhi-Instruct (MLX · 4-bit)
This repository contains an MLX-format 4-bit quantized version of
microsoft/MediPhi-Instruct,
converted using mlx-lm for efficient on-device inference on Apple silicon.
This model is intended for iOS / iPadOS / macOS usage where memory and power constraints require aggressive quantization while preserving clinical reasoning quality.
Model details
- Base model: MediPhi-Instruct (Phi-3 family)
- Parameters: ~3.8B
- Quantization: 4-bit (MLX)
- Format: MLX (not GGUF)
- Intended use: On-device medical and clinical QA, decision support, and explanations
- Language: English
⚠️ This is a conversion only. No additional fine-tuning was performed.
Why MLX 4-bit?
Compared to larger 4–7B medical models, MediPhi-Instruct shows:
- Strong clinical reasoning per parameter
- Better robustness under 4-bit quantization
- Lower memory footprint suitable for mobile devices
This makes it a strong candidate for on-device medical assistants on iPhone and iPad.
Usage (MLX-LM)
Install
pip install mlx-lm
- Task
- Text Generation
- Library
- mlx
- Type
- phi3
- License
- mit
- Languages
- en
- Created
- Dec 29, 2025
- Updated
- Dec 30, 2025