Laguna-S-2.1-GGUF-ROCMFPX

Laguna-S-2.1 — ROCmFPX hybrid quant (Q4 experts + Q8 rest)

This is a quantization of unsloth/Laguna-S-2.1-GGUF (BF16) (the original model is Poolside's Laguna S 2.1, a 118B-total / 8B-activated MoE with 256 routed experts).

[!IMPORTANT] You need the ROCmFPX fork of llama.cpp (or a llama.cpp build with ROCmFPX support). This file uses the experimental q4_0_rocmfp4_fast (type 101) and q8_0_rocmfpx (type 103) weight formats, which stock llama.cpp releases do not understand — loading it elsewhere will fail with an unknown tensor type error.

Recipe

• Total size: ~61.6 GB (4.39 bpw) vs 224 GB BF16 source — single merged file, no shards
• Rationale: routed experts dominate parameters (~97%) and tolerate 4-bit well; everything else stays near-lossless at 8-bit.

How it was made

# from the ROCmFPX fork (CPU-only build works fine for quantization)
llama-quantize \
  --tensor-type "ffn_(gate|up|down)_exps=q4_0_rocmfp4_fast" \
  Laguna-S-2.1-BF16-00001-of-00005.gguf \
  Laguna-S-2.1-Q8_0_ROCMFPX-Q4FAST-experts.gguf Q8_0_ROCMFPX
# then merged from 5 shards: llama-gguf-split --merge ...

Note the leading dense layer (blk.0) keeps its dense FFN at q8_0_rocmfpx — only the routed expert tensors were overridden.

Usage

# build ROCmFPX for your GPU (see the repo README; e.g. Strix Halo):
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh

# run (Vulkan was fastest in upstream tests on Strix Halo):
./build-strix-rocmfp4/bin/llama-cli \
  -m Laguna-S-2.1-Q8_0_ROCMFPX-Q4FAST-experts.gguf \
  -dev Vulkan0 -ngl 999 -fa on --jinja

Benchmarks

[!NOTE] Benchmarks are pending — placeholder table below.

Quality comparison vs BF16 source (perplexity / KLD): TBD.

Attribution & license

• Quantized from: unsloth/Laguna-S-2.1-GGUF (BF16 shards)
• Original model: Poolside Laguna S 2.1
• License: openmdw-1.1 (inherited from the source model)
• Quant formats by the ROCmFPX project