Laguna-S-2.1-MLX-3bit

Laguna-S-2.1-MLX-3bit

MLX (Apple Silicon) conversion of poolside/Laguna-S-2.1, quantized to 3-bit (affine group quant, group size 64; router gate kept at 8-bit).

Code / loader: github.com/PipeNetwork/laguna-mlx

Laguna-S 2.1 is Poolside's 118B-total / ~8B-active sparse-MoE model (256 routed experts, top-10, + 1 shared expert), built for agentic coding and long-horizon work with a 1M-token context. It interleaves full-attention and sliding-window (512) layers, uses softplus per-head attention gating, per-head Q/K RMSNorm, and a sigmoid router with an aux-loss-free correction bias.

Quantizations

⚠️ Loading requires the bundled laguna.py loader

The laguna architecture is not in stock mlx-lm, so this repo bundles the numerically-validated MLX implementation as laguna.py. Register it once, then use mlx-lm normally:

pip install mlx mlx-lm
python - <<'PY'
import os, shutil, mlx_lm
from huggingface_hub import hf_hub_download
dst = os.path.join(os.path.dirname(mlx_lm.__file__), "models", "laguna.py")
shutil.copy(hf_hub_download("pipenetwork/Laguna-S-2.1-MLX-3bit", "laguna.py"), dst)
print("registered laguna ->", dst)
PY

mlx_lm.generate --model pipenetwork/Laguna-S-2.1-MLX-3bit \
    --prompt "Write a Rust function that reverses a linked list." --max-tokens 256

Needs an Apple-Silicon Mac with enough unified memory to hold the weights (≈ the size above).

Notes

• Quantized: attention / MLP / expert projections and the token embedding + LM head. Kept at higher precision: the MoE router gate (8-bit), RMSNorms, and the aux-loss-free correction bias.
• Sliding-window layers use a rotating KV cache (bounded at 512), so long-context decode stays memory-bounded.
• Dual RoPE is reproduced exactly: full-attention layers use partial-rotary (0.5) YaRN (theta 5e5, factor 128, mscale 1.4852); sliding layers use plain RoPE (theta 1e4, full rotary).

License: OpenMDW-1.1 (inherits the base model).