maple-preview-20b-q4nx

Maple-Preview 20B (Q4NX & TQ2_0 Quantized for FastFlowLM / AMD Ryzen AI NPU)

This repository contains the FastFlowLM Q4NX & TQ2_0 quantized weights of DeepGrove's Maple-Preview (20B-A1B ternary reasoning MoE model), optimized for high-speed inference on AMD Ryzen AI NPUs (XDNA 2 / Strix Point) and modern AVX2/FMA CPUs.

🚀 Model & Runtime Highlights

• Base Architecture: 20B Total Parameters (~1B active per token, 256 experts with Top-8 routing).
• Hybrid 3:1 Attention: 18 Sliding Window Attention layers (512-token span) + 6 Full Global Attention layers.
• Native 128K Context Window: Full 131,072 token context support with $O(1)$ constant 18.9 MB SWA memory overhead.
• Hardware NPU Acceleration: Achieves >185–202 tokens/second on AMD Ryzen AI 9 HX 370 NPU (/dev/accel/accel0).
• OpenAI-Compatible REST Server: Native /v1/chat/completions and streaming <think> reasoning support.

---

📦 File Inventory

• model.q4nx: FastFlowLM packed ternary/quantized weights for NPU & CPU runtime.
• maple-preview-TQ2_0-head-Q4_K.gguf: GGUF format checkpoint for cross-runtime compatibility.
• config.json: Architecture configuration and hyperparameter specifications.
• tokenizer.json & tokenizer_config.json: Byte-level BPE tokenizer (151,936 vocabulary).
• chat_template.jinja: Jinja chat template with reasoning toggle (<think> ... </think>).

---

🛠️ Quickstart with FastFlowLM

1. Clone Runtime & Setup

git clone https://github.com/phantomic12/maple-flm.git
cd maple-flm
bash install.sh

2. Download Weights via Hugging Face CLI (hf)

hf download phantomic12/maple-preview-20b-q4nx --local-dir models/maple-preview-20b

3. Run Inference on NPU

# Interactive Chat
bash scripts/chat_maple.sh

# Launch OpenAI-Compatible REST Server
bash scripts/serve_maple.sh

# Run Hardware Benchmarks
bash scripts/benchmark_npu.sh

---

📊 Performance Benchmarks (AMD Ryzen AI 9 HX 370 NPU)

---

📜 Citation & Credits

• Original Model: deepgrove/maple-preview
• FastFlowLM Runtime: ROCm/FastFlowLM
• Port & Optimizations: phantomic12/maple-flm