Downloads · 30 days
21
9% of all-time downloads
groxaxo/Laguna-S-2.1-UD-IQ4_S-GGUF
Laguna-S-2.1-UD-IQ4_S-GGUF is a machine learning model from groxaxo. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Laguna-S-2.1-UD-IQ4 S-GGUF is a GGUF release for llama.cpp-compatible runtimes and local inference, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible local o…
Downloads · 30 days
21
9% of all-time downloads
All-time downloads
238
Public
Repo size
70 GB
Likes
1
Public
Click a slice to open those files.
.gguf70 GB · 100%
From the Hugging Face model README
Laguna-S-2.1-UD-IQ4 S-GGUF is a GGUF release for llama.cpp-compatible runtimes and local inference, published by groxaxo.
It is intended for open-source evaluation, reproducible experimentation, and compatible local or
hosted inference workflows. The wording below is deliberately limited to what can be verified
from this repository's metadata and artifacts.
| Field | Details |
|---|---|
| Format | GGUF |
| Source / base | poolside/Laguna-S-2.1 |
| Intended task | the task described by the included configuration and documentation |
| License | the license declared in the repository files |
*.gguf (5 files)Download a .gguf file that fits your available memory, then run it with a current llama.cpp
build:
llama-cli \
-m /path/to/model.gguf \
-p "Write a concise technical summary."
For vision or any-to-any models, download the matching multimodal projection file when one is provided and follow the source model's modality-specific instructions.
Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.
Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.
<!-- polished-overview:end -->Custom Unsloth Dynamic quantization of Poolside Laguna-S-2.1, optimized for 3× RTX 3090 (24 GB VRAM each) inference with llama.cpp.
This is an intermediate recipe between Unsloth's UD-Q4_K_S and UD-IQ4_NL,
tuned specifically for 3× 3090 setups where:
Built with llama-quantize using Unsloth's importance matrix (imatrix_unsloth.gguf)
and a per-tensor type override file.
| Precision | Tensors | Assignment |
|---|---|---|
| Q2_K | 386 | Base experts + attention (shared with UD-Q4_K_S) |
| F32 | 287 | Norms, scales, gate inputs |
| Q5_K | 69 | ffn_down_exps (all 45) + ffn_up_exps layers 1-24 |
| IQ4_XS | 70 | ffn_gate_exps (all 47) + ffn_up_exps layers 25-47 |
| IQ2_XXS | 2 | blk.46.ffn_down_exps + blk.47.ffn_down_exps (last layers) |
| Total | 814 | 256×4.5B MoE, 48 layers |
BPW: 4.76 | Size: ~67 GB | Compression: 3.3× from BF16
ffn_down_exps (output projection, most quality-sensitive) → Q5_K (higher precision)ffn_gate_exps (gating, least sensitive) → IQ4_XS (more compressed)ffn_up_exps split by layer depth — early layers (1-24) get Q5_K, late layers (25-47) get IQ4_XSffn_down_exps → IQ2_XXS (minimal quality impact, saves space)unsloth_calibration_Laguna-S-2.1.txtllama-server \
-m Laguna-S-2.1-UD-IQ4_S-00001-of-00005.gguf \
-ngl 999 \
-ts 1,1,1 \
-c 8192 \
--host 0.0.0.0 --port 8080
All 48 layers + 256 experts offload to GPU. Fits comfortably in 72 GB total VRAM with room for context.
imatrix_unsloth.gguf from the same repoOpenMDW 1.1 (same as base model)