Laguna-XS-2.1-6bit-XL-mlx

leonsarmiento/Laguna-XS-2.1-6bit-XL-mlx

This model was converted to MLX format from poolside/Laguna-XS-2.1 using BaseQuant_XL 6/8-bit mixed quantization. Text-only (no vision encoder). The unquantized BF16 base was used as the source.

BaseQuant_XL keeps the most routing-critical layers in full bf16 precision — the MoE router (mlp.gate.proj), the shared expert, and lm_head — while applying 8-bit to embeddings and all attention layers, and 6-bit to the 256 routed experts (switch_mlp). This is the recipe that consistently produces more decisive reasoning chains and fewer degenerate self-verification loops across our benchmark suite.

Laguna XS 2.1 is a 33B-parameter MoE model (3B active per token) built for agentic coding and long-horizon software engineering. It features a novel architecture with per-head gating, mixed sliding-window/global attention (3:1 ratio), and dual RoPE configurations. Native interleaved reasoning with preserved thinking is supported via enable_thinking.

Comparison: The XL recipe keeps routing at bf16 vs the AtomicChat uniform 6-bit which quantizes everything to 6-bit (router at 8-bit). Benchmark comparison below shows whether XL dominates on this architecture.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

Intelligence Benchmarks

Run under identical conditions on the same hardware. Instruct mode (thinking disabled), same sample sizes as our Gemma-4 26B 6bit-XL benchmark suite.

Instruct-mode benchmarks

Poolside published results (BF16, for reference)

These are poolside's published scores for the full-precision base model using their agent harness (Harbor Framework, max 500 steps, sandboxed execution).

Use with mlx

Requires mlx-vlm 0.6.3+ (not mlx-lm — Laguna's architecture is only in mlx-vlm):

pip install -U mlx-vlm

# Generate (reasoning on by default)
python -m mlx_vlm generate \
    --model leonsarmiento/Laguna-XS-2.1-6bit-XL-mlx \
    --prompt "Write a Python retry wrapper with exponential backoff." \
    --max-tokens 512 --temperature 1.0 --top-k 20 --top-p 1.0

# OpenAI-compatible server
python -m mlx_vlm server \
    --model leonsarmiento/Laguna-XS-2.1-6bit-XL-mlx \
    --host 0.0.0.0 --port 8080

LM Studio / oMLX

Load the model directory directly. Reasoning is native and on by default — use enable_thinking to control it.

BaseQuant_XL Quantization Strategy

Quantization Details

Architecture

Recommended Inference Parameters

Per poolside's benchmark settings:

Reasoning Control

Preserved Thinking

Laguna XS 2.1 is designed to work with preserved thinking — reasoning_content from prior assistant messages is kept in the conversation history so the model can reference its own earlier reasoning across turns. This is critical for multi-step agentic workflows where the model reasons between tool calls.

How it works: Unlike Qwen3.6 (which needs preserve_thinking=true) or Gemma 4 (which only preserves thinking for tool-calling turns), Laguna's template renders prior reasoning unconditionally when enable_thinking=true — no extra flags needed. The template extracts reasoning_content (or reasoning) from each assistant message and renders it inside <think>...</think> tags.

LM Studio / oMLX: Preserved thinking works automatically — no special configuration beyond enable_thinking=true. The server passes reasoning_content from prior assistant messages back to the template, which renders them into the prompt context.

Why it matters: Per poolside's documentation: "Reasoning may not be generated in follow-up steps if prior thinking blocks are dropped (i.e., thinking is not preserved) when messages are reconstructed over multiple steps." Without preserved thinking, the model loses its reasoning context and may produce inconsistent or degraded responses in multi-turn conversations.

License

OpenMDW-1.1 — permits free use, modification, and redistribution with attribution. This is an unofficial community quantization, not endorsed by poolside. Original LICENSE.md and notices of origin retained.