Downloads · 30 days
16
13% of all-time downloads
SmolLM2-135M-Instruct-MLX-5bit
Irfanuruchi/SmolLM2-135M-Instruct-MLX-5bit
SmolLM2-135M-Instruct-MLX-5bit is a text generation model from Irfanuruchi. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
This is an MLX conversion of HuggingFaceTB/SmolLM2-135M-Instruct quantized to 5-bit for fast on-device inference on Apple Silicon.
All-time downloads
119
Public
Parameters
135M
92.6 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors92.6 MB · 96%
Parameter types
How the weights are stored.
U32134M · 100%
Try a prompt
Base models
Model card
From the Hugging Face model README
SmolLM2-135M Instruct (MLX, 5-bit)
This is an MLX conversion of HuggingFaceTB/SmolLM2-135M-Instruct quantized to 5-bit for fast on-device inference on Apple Silicon.
Quickstart
Install:
pip install -U mlx-lm
Run:
mlx_lm.generate \
--model Irfanuruchi/SmolLM2-135M-Instruct-MLX-5bit \
--prompt "Reply with exactly 3 bullet points, 4–8 words each: what can you do offline?" \
--max-tokens 80
Benchmarks (MacBook Pro M3 Pro)
- Disk: 92 MB
- Peak RAM: 0.122 GB
Performance will vary across devices and prompts.
Notes
- Converted/quantized with
mlx_lm.convert. - This repo contains MLX weights and tokenizer/config files.
License & attribution
Upstream model: HuggingFaceTB/SmolLM2-135M-Instruct (Apache-2.0).
Please follow the upstream license and attribution requirements.
- Task
- Text Generation
- Library
- mlx
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Jan 24, 2026
- Updated
- Jan 24, 2026