Qwen3-4B-Instruct-2507-Power

Qwen3-4B-Instruct-2507 — Q4_K_M for IBM Power (Linux ppc64le + AIX)

Qwen3-4B-Instruct-2507 quantized to Q4_K_M with a Q6_K output head for fast, accurate CPU inference on IBM Power — POWER9 (VSX) and POWER10/11 (MMA-accelerated) via LibrePower. No GPU required. Size: 2.4G.

Run it

Ubuntu / Debian ppc64le:

curl -fsSL https://linux.librepower.org/install.sh | sudo sh
sudo apt install librepower-llama
wget https://huggingface.co/librepowerai/Qwen3-4B-Instruct-2507-Power/resolve/main/Qwen3-4B-Instruct-2507-Q4_K_M.gguf
lp-llama-completion -m Qwen3-4B-Instruct-2507-Q4_K_M.gguf -p "Hello!" -n 64 -t $(nproc)

IBM AIX 7.3 (big-endian):

dnf install llama-aix
wget https://huggingface.co/librepowerai/Qwen3-4B-Instruct-2507-Power/resolve/main/Qwen3-4B-Instruct-2507-Q4_K_M-be.gguf
lp-llama-completion -m Qwen3-4B-Instruct-2507-Q4_K_M-be.gguf -p "Hello!" -n 64 -t $(nproc)

Files

• Qwen3-4B-Instruct-2507-Q4_K_M.gguf — little-endian (Ubuntu/Linux ppc64le)
• Qwen3-4B-Instruct-2507-Q4_K_M-be.gguf — big-endian (IBM AIX)

Good for

Best all-rounder for its size: chat, code, reasoning, 256K context

Credits

Base model by its original authors (Apache-2.0). Quantization & Power packaging: LibrePower.