Downloads · 30 days
26
18% of all-time downloads
amanwalksdownthestreet/Laguna-S-2.1-exl3
Laguna-S-2.1-exl3 is a text generation model from amanwalksdownthestreet. Use it when you need the model to write or continue text. The card lists the license as openmdw-1.1.
Optimized exllama quantizations of poolside/Laguna-S-2.1 (118B-A8B, 1M context).
Downloads · 30 days
26
18% of all-time downloads
All-time downloads
145
Public
Repo size
644 GB
Likes
0
Public
Click a slice to open those files.
.json2.4 MB · 100%
From the Hugging Face model README
Optimized exllama quantizations of poolside/Laguna-S-2.1 (118B-A8B, 1M context).
Quantized with exllamav3 v1.2.0.
Head layer retained at full precision, attention layers across all optimized quants pinned to 6.0bpw (nearly free on this model — worth ≈+1bpw of effective fidelity for ≈0.05bpw of size, measured against an unboosted twin), measured with 34 calibration rows to account for all of the tiny experts in this model at l 3.
The exllamav3/QTIP optimization ceiling for this model appears to be ≈5.0bpw: a marginal-returns reconstruction of the measurement shows the 4.80 branch already captures >99.9% of the achievable fidelity improvement, and as such 4.80bpw-opt-attn-h16 is the recommended quant, VRAM allowing. 5.29 adds ≤0.1% measurable fidelity for 6.6 GB.
Additional quants can be generated by anyone using my measurement data in measurement.json and the base quants.
| Branch | Size | HumanEval | MBPP+ | LCB v6 | KLD vs BF16 | top-1 |
|---|---|---|---|---|---|---|
| 2.36bpw-opt-attn-h16 | 33.3 GB | 85.4 | 87.6 | 34.7 | 0.455 | 71.5 |
| 2.80bpw-opt-attn-h16 | 39.4 GB | 89.6 | 90.0 | 41.5 | 0.374 | 74.0 |
| 3.55bpw-opt-attn-h16 | 49.6 GB | 90.2 | 90.5 | 33.9† | 0.321 | 76.1 |
| 4.15bpw-opt-attn-h16 | 57.8 GB | 90.2 | 89.7 | 40.5 | 0.302 | 76.6 |
| 4.80bpw-opt-attn-h16 | 66.6 GB | 92.1 | 89.4 | 42.6 | 0.293 | 77.0 |
| 5.29bpw-opt-attn-h16 | 73.2 GB | 92.1 | 88.6 | 41.6 | 0.286 | 77.5 |
HumanEval / MBPP+: pass@1, greedy decoding (t=0), thinking disabled, via lm-eval humaneval_instruct / mbpp_plus_instruct (MBPP+ with a continuation-safe extraction filter). n=164 / n=378; stderr ≈ ±2.1 / ±1.6. Saturated for branches ≥2.80bpw; exact ties between adjacent branches are coarse-metric collisions (per-problem outputs differ).
LCB v6: LiveCodeBench code_generation_lite v6 window — 175 problems, 2025-01 → 2025-04, 46% hard. Upstream config (n=10, t=0.2, top_p=0.95), max_tokens 25000, thinking disabled, LCB @ 28fef95, served on TabbyAPI/exllamav3. pass@1; stderr ≈ ±3.5. †Within noise of neighbors — fidelity column confirms the branch is healthy. The quality cliff sits between 2.80 and 2.36; LCB cannot rank branches above the cliff at this sample size, KLD can.
KLD vs BF16 / top-1: exllamav3 model_diff.py against the BF16 reference over 10 held-out 2048-token sequences (20,480 positions) — mean KL divergence (quant‖reference) and top-1 token agreement (%). Strictly monotone in bpw; the fidelity curve knees between 3.55 and 2.80, independently confirming the LCB cliff. Do not rank quants by perplexity: low ppl ≠ close to the reference.
Terminal-Bench 2.1 (5.29 branch): 50.6% (44/87) via terminus-2, agent-timeout ×6.0, verifier ×4.0, served locally. Not comparable to poolside's published 70.2 — different agent scaffold, and the scaffold is half the score.
| Branch | Size |
|---|---|
| 2.0bpw-h16 | 28.8 GB |
| 3.0bpw-h16 | 42.4 GB |
| 4.0bpw-h16 | 56.0 GB |
| 5.0bpw-h16 | 69.6 GB |
| 6.0bpw-h16 | 83.3 GB |