vibethinker-3b-mxfp4-mlx

vibethinker-3b-mxfp4-mlx

MLX quantization of WeiboAI/VibeThinker-3B for Apple Silicon.

Variant: Block float MX FP4
Disk size: 1575 MB
Quantized by: sahilchachra

Benchmark results

Evaluated on Apple M5 Pro with MLX. Model loaded once; performance and quality measured in a single pass.

Performance

Warmed, short-prompt, chat-templated, thinking disabled. Represents steady-state decode for typical chat use; long thinking traces will be slower due to KV-cache growth.

Quality

MATH-500 per-level accuracy

Context scaling (decode tok/s)

Prompt format & recommended settings

VibeThinker-3B is a reasoning model (built on Qwen2.5-3B). It thinks inside a <think>…</think> block and then writes its final answer, so the way you call it matters:

• Always apply the chat template (tokenizer.apply_chat_template(..., add_generation_prompt=True)). Passing a raw string skips the <|im_start|> / <|im_end|> markers the model was trained on.
• Give it room to think. It can emit thousands of reasoning tokens before the answer — use max_tokens of at least 8192 (16384+ for hard competition math). Small caps cut it off mid-thought.
• For math, ask for the final answer in \boxed{} — that's how it was trained and how the benchmarks below were scored. Parse the text after </think> for the answer.
• Context length: up to 64K tokens.
• Stop token: these MLX variants set eos_token_id to include <|im_end|> (151645) so generation halts cleanly at the turn boundary.

Recommended sampling (from the original model card): temperature=1.0, top_p=0.95, top_k disabled.

The benchmark numbers above were produced with greedy decoding (temperature 0) for reproducibility — so a variant's score reflects quantization damage, not sampling noise. For everyday use, the recommended sampling settings give better, more diverse reasoning.

Usage

pip install mlx-lm

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("sahilchachra/vibethinker-3b-mxfp4-mlx")

messages = [{"role": "user",
             "content": "Let x be the number of ways to ... . Give the final answer in \\boxed{}."}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False,
)

# Reasoning needs a large token budget; sampling per the model card.
sampler = make_sampler(temp=1.0, top_p=0.95)
response = generate(
    model, tokenizer, prompt=prompt,
    max_tokens=16384, sampler=sampler, verbose=True,
)
# `response` contains a <think>...</think> trace followed by the final answer.

All variants in this collection

Notes

• Requires Apple Silicon (M1 or later) with MLX
• Benchmarks run on Apple M5 Pro, 24 GB unified memory
• License: see WeiboAI/VibeThinker-3B for the original model's license

Original model

See WeiboAI/VibeThinker-3B for full model details and intended use.