Laguna-S-2.1-Q4_0-ROCMFP4-GGUF

Laguna-S-2.1 — Q4_0 ROCmFP4 GGUF (experimental, Strix Halo / gfx1151)

An experimental 4-bit quantization of poolside/Laguna-S-2.1 (118B MoE, 1M context). Built for and validated on a single AMD Strix Halo APU - Ryzen AI Max+ 395, gfx1151, 128 GB unified memory.

[!IMPORTANT] This file does not run on upstream llama.cpp, Ollama, LM Studio, or vLLM. It uses the custom Q4_0_ROCMFP4_STRIX_LEAN tensor layout and requires a ROCmFPX build as described below. Unsupported runtimes should reject the file; if a tool appears to load it anyway, do not trust the output.

[!WARNING] Validated on gfx1151 (Strix Halo) only. On RDNA4 (gfx1201, Radeon AI PRO R9700) this artifact hung during inference in both HIP and Vulkan attempts on the test rig. Treat it as unsupported there until that's resolved. RDNA4 users should use Q4_0_ROCMFP4_FAST instead.

Should you use this?

Use this model if all of the following are true:

• you run a Strix Halo machine (Ryzen AI Max+ 385/395, gfx1151) with 96 GB+ of unified memory allocatable to the GPU;
• you are willing to build the ROCmFPX fork from source;
• you want the fastest measured batch-1 decode for Laguna-S-2.1 on this hardware class.

Otherwise, use an ordinary K-quant GGUF such as unsloth's UD-Q4_K_XL. It runs everywhere, and on this same test machine it decodes at 29.5 t/s vs 41.5 t/s for this file.

TL;DR — measured on one Strix Halo machine

Decode is +41% over the K-quant control on identical hardware, engine generation, sampling, and prompt. Prefill is ~12% slower at this shallow depth. Runs used streaming chat completions with prompt caching disabled (cold prefill every run), best-of-3. Public run: LocalMaxxing cmryiee3u03udo401ztqz7dde.

What was NOT measured: output quality. No HumanEval, MBPP, perplexity, or KL-divergence comparison against the F16 source has been run for this artifact. The STRIX_LEAN recipe keeps attention K/V tensors and token embeddings at higher precision (Q5_K embeddings) to protect quality. But until evals are published here, treat quality parity as unverified.

What Q4_0_ROCMFP4_STRIX_LEAN actually is

A 4.38 bit-per-weight layout from the ROCmFPX project. The bulk of the network uses the Q4_0_ROCMFP4_FAST single-scale FP4 block layout. Attention K/V projection tensors follow the Strix-tuned quality recipe, and token embeddings are kept at Q5_K. The recipe was tuned on gfx1151 (ROCmFPX PR #35, merged 2026-07-23). This is a memory-layout and kernel-path optimization, not native FP4 matrix hardware arithmetic.

The quantization is uniform with no importance matrix.

Required runtime

git clone https://github.com/charlie12345/ROCmFPX.git
cd ROCmFPX && git checkout main   # validated at commit c190e43

# Strix Halo standard build (HIP + Vulkan):
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh    # -> build-strix-rocmfp4/

The test rig's HIP toolchain wasn't working, so the sealed numbers came from a Vulkan-only build:

cmake -B build-strix-vulkan -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release \
      -DLLAMA_BUILD_WEBUI=OFF
cmake --build build-strix-vulkan -j16 --target llama-server

Example deployment (as measured)

./build-strix-vulkan/bin/llama-server \
  -m Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  -dev Vulkan0 \
  -ngl 999 \
  --flash-attn on \
  --jinja \
  -ctk f16 -ctv f16 \
  -b 512 -ub 128 -t 16 \
  -np 1 \
  -c 65536

-b 512 -ub 128 is a decode-oriented shape; it costs prefill throughput. For long-prompt/agentic use, raise -ub and re-measure.

Measurement conditions

• Framework Desktop, AMD Ryzen AI Max+ 395 (Strix Halo, gfx1151), 128 GB unified memory, Arch Linux, RADV Vulkan driver.
• ROCmFPX main at commit c190e43; control ran the same day's upstream llama.cpp (b10103-4-gc0bc859) with the same sampling and flags apart from quant-specific paths.
• Best-of-3 streaming requests, cache_prompt: false, 564-token prompt, 256-token generation, one model resident at a time, no other GPU load.

Artifact

Quantization command (ROCmFPX c190e43):

./build-r9700/bin/llama-quantize \
  laguna-s-2.1-F16.gguf \
  Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN \
  12

Quantization was done on the R9700 build host, then the file was transferred to Strix Halo for measurement. The build-r9700 path is just the build directory name from that host.

Verify after download:

sha256sum -c SHA256SUMS

Limitations

• Requires the ROCmFPX fork; no upstream llama.cpp compatibility.
• Throughput-validated only. No quality evals published for this artifact yet.
• Validated on exactly one Strix Halo host, Vulkan/RADV, batch 1, shallow prompt depth. Deeper contexts, HIP on Strix, and batch >1 are unmeasured.
• Known bad on RDNA4 gfx1201 (inference hang, under investigation upstream).
• MoE 118B: needs ~60 GB for weights plus KV. 64 GB-class machines can't run it with meaningful context.

License and attribution

• Base model: Laguna-S-2.1, Copyright 2026 poolside, OpenMDW-1.1. This repository redistributes a converted and quantized derivative under the same license.
• Format and execution path: the Q4_0_ROCMFP4_STRIX_LEAN representation and kernels are the work of the ROCmFPX project, which builds on llama.cpp.
• This repository: the quantized artifact and the Strix Halo measurement.

Laguna and related marks belong to their owners. This community quantization is not affiliated with or endorsed by poolside, AMD, ROCmFPX, or llama.cpp.