Downloads · 30 days
14
19% of all-time downloads
Qwen3-16B-A1.5B
SamPurkis/Qwen3-16B-A1.5B
Qwen3-16B-A1.5B is a machine learning model from SamPurkis. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A man-made horror beyond your comprehension.
All-time downloads
73
Public
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors32.1 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- qwen3_moe
- Access
- Public
- Created
- Jun 5, 2025
- Updated
- Jun 5, 2025
- SHA
- b619da09
Base models
Model card
From the Hugging Face model README
Qwen3-16B-A3B
A man-made horror beyond your comprehension.
But no, seriously, this is my experiment to:
- measure the probability that any given expert will activate (over my personal set of fairly diverse calibration data), per layer
- prune 64/128 of the least used experts per layer (with reordered router and indexing per layer)
It can still write semi-coherently without any additional training or distillation done on top of it from the original 30b MoE. The .txt files with the original measurements are provided in the repo along with the exported weights.
Custom testing to measure the experts was done on a hacked version of vllm, and then I made a bespoke script to selectively export the weights according to the measurements.
- Type
- qwen3_moe
- License
- apache-2.0
- Created
- Jun 5, 2025
- Updated
- Jun 5, 2025