Qwythos-9B-v2-MXFP8

Qwythos-9B-v2 — MXFP8 (MLX)

MXFP8 (8-bit microscaling, group-32, E8M0 shared exponents) quantization of empero-ai/Qwythos-9B-v2 for Apple Silicon via MLX / mlx-lm. ~8.6 GB on disk, 8.25 bits/weight (higher fidelity than 4-bit; needs a bit more memory).

Qwythos-9B-v2 is a 9B reasoning model on the Qwen3.5 dense hybrid architecture (3:1 Gated-DeltaNet linear attention : full attention).

Text-only — by design

The base model carries a Qwen3.5 multimodal stack, but the original author's card says the "multimodal stack is text-only in practice" and explicitly recommends running with multimodal inputs disabled:

--limit-mm-per-prompt '{"image":0,"video":0}'

i.e. image:0, video:0. Qwythos is a text reasoning model. In keeping with that, this quant is built with mlx-lm (text path) and smoke-tested with mlx-lm — it works (real samples + tok/s below). Vision is intentionally not included; it is neither claimed nor intended by the author. (As an aside, the current mlx-vlm 0.6.4 vision wrapper also mis-handles this architecture and emits garbage, so a vision build would be unusable regardless — another reason the text-only mlx-lm path is the correct one here.)

Quantization

• python -m mlx_lm convert --hf-path empero-ai/Qwythos-9B-v2 --q-mode mxfp8
• MXFP8 (8-bit microscaling, group-32, E8M0 shared exponents); 8.25 bpw effective. Linear layers quantized; norms/embeddings kept higher-precision by MLX (standard behavior).

✅ Verified end-to-end (tested 2026-07-13, Apple M-series)

Real output from this quantized model, generated with mlx-lm — not a claim. Average 32.1 tok/s across 4 prompts, peak memory ~9.4 GB (batch 1, greedy-ish at temp 0.6). Coherent step-by-step reasoning confirmed.

Samples

Prompt: In one sentence, what is a dependent type?

The user asks: "In one sentence, what is a dependent type?" I need to provide a concise definition that captures the essence of dependent types.

First, recall what dependent types are: they are a feature in programming languages where the type of an expression can depend on the value of another expression. This allows for stronger guarantees about code correctness because types can encode properties of values.

I must ensure the definition is accurate and fits into a single sentence. Let me draft: "A dependent type is one whose structure depends on the values of other expressions." That seems clear and directly answers the question. It mentions that the type's structure (or definition) is not static but varies based on other expressions' values, which is the core idea.

Check if any nuanc …

220 tok @ 31.7 tok/s

---

Prompt: Write a Python function that returns True if a string is a palindrome, ignoring case and spaces.

I need to write a Python function that checks if a given string is a palindrome, ignoring case and spaces. Let's think about the requirements:

1. Palindrome definition: A string reads the same forwards and backwards.
2. Case insensitivity: 'A' should be treated the same as 'a'.
3. Space handling: Spaces should be ignored (removed) before checking.

My approach will be:

• Convert the input string to lowercase using str.lower().
• Remove all spaces using list comprehension or replace, or perhaps filter out any whitespace. Since the prompt specifically says "ignoring spaces", I'll just remove spaces.
• Then compare the cleaned string with its reverse.

I also need to consider edge cases:

• Empty string: Should return True (empty string is technically a palindrome). I could ha …

220 tok @ 32.1 tok/s

---

Prompt: A train travels 60 km in 45 minutes. What is its average speed in km/h? Show your reasoning.

Goal: Compute the train's average speed in km/h and present the reasoning clearly.

Given data:

• Distance traveled = 60 km
• Time taken = 45 minutes

Step 1: Convert time to hours. Since speed is requested in km/h, I must convert minutes to hours.

• 1 hour = 60 minutes
• 45 minutes = 45/60 hours = 3/4 hour (or 0.75 h)

Step 2: Apply the speed formula. Average speed = total distance / total time.

• Speed = 60 km / (3/4) h
• Dividing by a fraction is the same as multiplying by its reciprocal: 60 * (4/3) = 80 km/h.

Step 3: Present the reasoning. I should show each conversion and calculation explicitly so the user can follow the logic.

Drafting the response:

• State the formula.
• Show the conversion

220 tok @ 32.2 tok/s

---

Prompt: Explain step by step how to compute the factorial of 5, then give the result.

The user wants a step-by-step explanation of how to compute the factorial of 5, and then they want the final result.

First, I need to recall what a factorial is: the product of all positive integers up to that number. So, 5! = 5 × 4 × 3 × 2 × 1.

Now, I should outline each multiplication step clearly:

• Start with 5.
• Multiply by 4 → 5 × 4 = 20.
• Multiply the result (20) by 3 → 20 × 3 = 60.
• Multiply that (60) by 2 → 60 × 2 = 120.
• Multiply that (120) by 1 → 120 × 1 = 120.

Thus, the final answer is 120.

I should present this in a concise, step-by-step format that directly answers the user's request. No need to mention my identity unless asked

220 tok @ 32.3 tok/s

Usage (Apple Silicon + pip install mlx-lm)

CLI:

python -m mlx_lm generate --model sahilchachra/Qwythos-9B-v2-MXFP8 \
  --prompt "A train travels 60 km in 45 min. Average speed in km/h?" \
  --temp 0.6 --top-p 0.95 --top-k 20 --max-tokens 512

Python:

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler, make_logits_processors
model, tok = load("sahilchachra/Qwythos-9B-v2-MXFP8")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Your prompt"}], add_generation_prompt=True)
print(generate(model, tok, prompt, max_tokens=512,
               sampler=make_sampler(temp=0.6, top_p=0.95, top_k=20),
               logits_processors=make_logits_processors(repetition_penalty=1.05)))

Prompt template & sampling (mirrors the base model)

Qwen3.5 ChatML template (bundled; applied by apply_chat_template). Base-model recommended sampling: temperature 0.6, top_p 0.95, top_k 20, repetition_penalty 1.05. Context up to 1,048,576 tokens (YaRN factor 4).

Base-model benchmarks (reported by the original author, not re-measured here)

MMLU-CoT 83.8 · ARC-Challenge 96.4 · GPQA-diamond 49.0 · GSM8K 93.6 · HumanEval pass@1 77.4. See the base model card. Quantization can shift these; provided as reference for the un-quantized model.

Note

The base model is intentionally uncensored for research/technical use — deploy responsibly and within applicable law.

License

Apache-2.0, inherited from empero-ai/Qwythos-9B-v2. Community MLX quantization, not affiliated with the original author.