VibeThinker-1.5B-MLX-nvfp4

VibeThinker-1.5B-MLX-nvfp4

This repository contains the 4-bit NVFP4 quantized weights for WeiboAI/VibeThinker-1.5B, optimized for low-latency, edge-based deployment on Apple Silicon hardware using the oMLX framework.

VibeThinker-1.5B is a dense Transformer reasoning model created by Sina Weibo Inc. It is engineered to challenge traditional scaling laws using the Spectrum-to-Signal Principle (SSP)—combining Two-Stage Diversity-Exploring Distillation with MaxEnt-Guided Policy Optimization (MGPO) to extract deep math and coding capabilities from a tiny 1.5B parameter core.

---

⚡ Inference Generation Breakthroughs (vs. VibeThinker-3B-nvfp4)

When benchmarked on Apple Silicon via the oMLX inference engine, this ultra-compact 1.5B parameter NVFP4 quantization delivers staggering speedups and resource savings compared directly to its 3B NVFP4 sibling.

Core Efficiency Multipliers:

• 🏎️ Speed Jump (Token Generation): Output velocity increases by +84.6% in standard generation, skyrocketing to 454.2 tok/s (compared to the 3B variant's 246.0 tok/s).
• 📉 Massive VRAM Savings: Reduces peak VRAM footprint by -38.6%, requiring a mere 1.51 GB of memory (vs. 2.46 GB for the 3B model), making it trivial to run on base-tier Mac hardware.
• ⚡ Prefill Processing Acceleration: The prompt prefill rate surges by +30.3% under standard context lengths (pp TPS climbs from 3,659 tok/s to 4,768.5 tok/s). Under massive 4k context limits, prefill speeds leap by +60.3% to hit 10,039.5 tok/s.
• 🚀 Concurrent Scaling (4x Batching): Under continuous multi-request batching, token throughput pushes forward to an incredible 773.5 tok/s—outperforming the 3B batched configuration by +68.5%.
• ⏱️ Near-Instant Turnaround: Total end-to-end processing latency drops by -37.9%, fulfilling a full reasoning response cycle in just 0.497 seconds.

---

🛠️ Deployment & Execution Quickstart

To run this model, use an inference engine configured to process the optimized nvfp4 memory layout natively on Mac (such as oMLX).

Example running with oMLX

# Execute local evaluation benches natively using the optimized Auto engine pipeline:
omlx bench --model your-hf-username/VibeThinker-1.5B-MLX-nvfp4 --prompt "Integrate x^2 ln(x) dx step by step."

<img src="/bkideas/Qwen2.5-Coder-3B-MLX-nvfp4/resolve/main/benchmark.svg" alt="Benchmark table" width="100%" />