Laguna-S-2.1-NVFP4-0804

Laguna-S-2.1-NVFP4-0804

This is poolside's Laguna S 2.1 in full NVFP4, restored to a size that fits a single 96 GB GPU (one RTX PRO 6000 Blackwell, for example). Poolside's 2026-08-01 update to poolside/Laguna-S-2.1-NVFP4 added a rule that exempts the routed experts in layers 40 through 47 from quantization. Those 19.3B parameters ship as BF16, which grows the checkpoint from 71.9 GB to 99.7 GB and pushes it off every 96 GB card. This repo requantizes exactly those tensors back to NVFP4 with the same conventions the rest of the checkpoint already uses, and changes nothing else. The result is 71.9 GB, carries the update's weights, and keeps the 1M-token context configuration.

Lineage

1. poolside/Laguna-S-2.1 by poolside. 117.6B-parameter MoE, 8.5B active, 48 layers, 256 experts per sparse layer.
2. poolside/Laguna-S-2.1-NVFP4 at revision f8fdfcdc4e (2026-08-01) by poolside. The source for this repo: layers 0 through 39 quantized by poolside, layers 40 through 47 shipped in BF16.
3. This repo, by kkuspa. Layers 40 through 47 requantized to NVFP4 from the BF16 tensors in item 2. Every other tensor is copied from item 2 unchanged.

Two provenance facts matter. First, the BF16 expert tensors in the 08-01 release are byte-identical to the original release of poolside/Laguna-S-2.1, so this requantization operates on the same weights poolside's own exporter would have used. Second, the quantization procedure below was verified bit-exact against poolside's shipped bytes: quantizing a tensor that poolside also ships in quantized form reproduces their packed weights, group scales, and global scales exactly, byte for byte, on every sampled tensor including fused gate/up pairs.

Quantization recipe

compressed-tensors nvfp4-pack-quantized (W4A4), group size 16, matching the checkpoint's existing scheme:

# per tensor, observer math in FP32 (BF16 observer math does not reproduce poolside's bytes)
global_scale = 448.0 * 6.0 / amax(weight)            # FP8_E4M3_MAX * FP4_E2M1_MAX / absmax
# gate_proj and up_proj of each expert share one global scale, min(gs_gate, gs_up),
# because vLLM fuses them into a single w13 kernel; down_proj keeps a per-tensor scale
group_scale  = amax(group_of_16) / 6.0 * global_scale  # snapped to FP8_E4M3, stored as FP8
weight_packed = pack_fp4_to_uint8(quantize(weight, group_scale, global_scale))

Weight quantization is data-free (memoryless_minmax, matching the checkpoint). The per-projection activation global scales (input_global_scale), which do require calibration data, are taken from poolside's own 2026-07-22 checkpoint (revision 07614121b3), which quantized these same layers from the same underlying weights. Activations entering layers 40 through 47 are produced by layers 0 through 39, which are identical in both checkpoints, so the vendor calibration transfers.

Kept in BF16, unchanged from poolside's own layout: all attention projections, router gates, shared experts, the layer-0 dense MLP, and lm_head.

Serving (vLLM)

Smoke-tested on 1x RTX PRO 6000 Blackwell (96 GB), vLLM post-0.25 dev build with FlashInfer, NVFP4 CUTLASS kernels, FP8 KV cache:

vllm serve kkuspa/Laguna-S-2.1-NVFP4-0804 \
  --trust-remote-code \
  --speculative-config '{"method":"dflash","model":"poolside/Laguna-S-2.1-DFlash-NVFP4","num_speculative_tokens":7}' \
  --enable-auto-tool-choice \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1 \
  --max-num-seqs 32 \
  --max-model-len 32768 \
  --kv-cache-dtype fp8_e4m3 \
  --gpu-memory-utilization 0.90

The smoke tests behind this command served the local equivalent of this repo. Measured single-stream on the card above: about 125 tok/s on prose and 205 to 209 tok/s on code with the DFlash draft accepting 3.9 tokens per step, prefill about 20,500 tok/s on a 10K-token prompt. Use the current main revision of the DFlash draft repo; it is the one trained against this weight generation. The config ships the 1M context window; set --max-model-len to what your KV budget allows (32768 was the tested value at these settings).

Packaging notes

Tokenizer, chat template, modeling code, and generation config are copied from the source revision unchanged. The model.safetensors.index.json covers all 49 shards. The only config change is the removal of the single quantization_config.ignore rule that exempted layers 40 through 47.

Caveats

• This artifact reproduces the non-convergent reasoning behavior documented in discussion 16 on the poolside repo. Under that protocol (single-turn debugging task, thinking enabled, no speculative decoding, temperature 0.7, max_tokens 8192), 0 of 20 requests produced a final answer and 20 of 20 exhausted the token budget inside reasoning. The model identifies the correct answer early in the trace and then keeps re-deriving it without concluding. The same behavior was reported against poolside's 2026-07-22 all-NVFP4 revision, so it is not introduced by this requantization, but it is present here. Whether poolside's 08-01 release (which keeps the last 8 layers' experts in BF16 and does not fit 96 GB) behaves differently is untested as of this writing. A repetition penalty of 1.15 partially mitigates it per discussion 10.
• Quality validation is otherwise limited. An 80-prompt IFEval slice scored within sampling noise of poolside's official 2026-07-22 all-NVFP4 checkpoint under identical serving (0.45 vs 0.50, standard error 0.08). No full benchmark suite has been run.
• Absolute instruction-following scores on both checkpoints in that comparison were depressed by a known reasoning-parser issue that affects all Laguna serves (thinking content can leak into responses; see discussion 15 on the poolside repo). That issue is upstream of this repo and not introduced by it.
• Poolside has not stated why the 08-01 release left these layers in BF16. If it was a quality hedge for the final layers, this repo trades that hedge back for 27.8 GB. The IFEval comparison suggests the cost is small on that axis; whether it matters for the reasoning-convergence behavior in the first caveat is exactly the untested question.
• Long-context behavior beyond 32K was not tested here.

Attribution

Laguna S 2.1 is built and released by poolside under the OpenMDW-1.1 license, which permits modification and redistribution with attribution. This repo modifies quantization coverage only; all weights derive from poolside's releases. Quantized locally on a single RTX PRO 6000 Blackwell workstation.