Downloads · 30 days
70
1% of all-time downloads
sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx
Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx is a machine learning model from sahilchachra. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
MLX quantization of empero-ai/Qwythos-9B-Claude-Mythos-5-1M for Apple Silicon.
Downloads · 30 days
70
1% of all-time downloads
All-time downloads
5.8K
Public
Parameters
9B
9.3 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors9.2 GB · 100%
How the weights are stored.
U329B · 100%
From the Hugging Face model README
MLX quantization of empero-ai/Qwythos-9B-Claude-Mythos-5-1M for Apple Silicon.
Note — text tower only. The source model is a Qwen3.5-VL multimodal model (
Qwen3_5ForConditionalGeneration, with a vision encoder). This MLX conversion contains only the text/language tower — the vision encoder weights are not included, so this is a text-only model and does not accept image or video input. The text reasoning the original is benchmarked for (GSM8K, MMLU) is unaffected.It loads via the standard MLX LLM path (mlx-lm, LM Studio). For LM Studio compatibility the config carries
partial_rotary_factorinsiderope_parameters(LM Studio's engine hard-indexes that key, unlike mlx-lm which defaults it); the config is also tagged as a causal LM (architectures: ["Qwen3_5ForCausalLM"], vision/image/video token ids removed) to reflect that it is text-only.
Variant: Block float MX FP8
Disk size: 8826 MB
Quantized by: sahilchachra
Evaluated on Apple M5 Pro with MLX. Model loaded once; performance and quality measured in a single pass.
| This model | FP16 baseline | |
|---|---|---|
| Decode tok/s (avg, long traces) | 30.67 | N/A |
| Peak memory (GB) | 9.599 | N/A |
| Disk size (MB) | 8826 | 17969 |
| Benchmark | This model | FP16 baseline | n |
|---|---|---|---|
| GSM8K (math, accuracy) | 100.0% | N/A | 50 |
| MMLU (knowledge, accuracy) | 80.0% | N/A | 50 |
| Context length | Decode tok/s |
|---|---|
| ~128 tokens | 33.7 |
| ~256 tokens | 33.6 |
| ~512 tokens | 33.6 |
| ~1024 tokens | 33.5 |
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx")
response = generate(model, tokenizer, prompt="Your prompt here", max_tokens=256, verbose=True)
| Model | Variant |
|---|---|
| sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp4-mlx | Block float MX FP4 |
| sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx | Block float MX FP8 ← this model |
| sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-optiq-5bpw-mlx | OptiQ mixed-precision (target 5.0 bpw) |
See empero-ai/Qwythos-9B-Claude-Mythos-5-1M for full model details and intended use.