Laguna-S-2.1-DFLASH-GGUF

Base models

Model:
https://huggingface.co/unsloth/Laguna-S-2.1-GGUF

Dflash model:
https://huggingface.co/poolside/Laguna-S-2.1-GGUF

chat_template:
https://huggingface.co/poolside/Laguna-S-2.1-GGUF/raw/main/chat_template.jinja

Files

File manager:
https://huggingface.co/tepirale/Laguna-S-2.1-DFLASH-GGUF/blob/main/laguna_manager.sh

Config models:
https://huggingface.co/tepirale/Laguna-S-2.1-DFLASH-GGUF/blob/main/models.json

Auto model | GPU L40 '38 GiB / 45 GiB' V-Ram

!chmod +x laguna_manager.sh

# install llama.cpp or vllm
./laguna_manager.sh install llamacpp


# download Laguna-S-2.1-GGUF
hf download unsloth/Laguna-S-2.1-GGUF \
  --include "Laguna-S-2.1-UD-IQ1_M.gguf" \
  --local-dir ~/.laguna_manager/models/laguna-s-2.1-GGUF


# download Laguna-S-2.1-DFlash-GGUF
hf download poolside/Laguna-S-2.1-GGUF \
  --include "*DFlash*.gguf" \
  --local-dir ~/.laguna_manager/models/laguna-s-2.1-DFlash


# config
./laguna_manager.sh config set '.model="laguna-s-2.1" | .backend="llamacpp" | .quant="UD-IQ1_M" | .server.llamacpp.ctx_size=32768 | .server.llamacpp.spec_decoding=true'

# return config
'''
Actualizado.
{
  "backend": "llamacpp",
  "model": "laguna-s-2.1",
  "quant": "UD-IQ1_M",
  "vllm_variant": "fp8",
  "server": {
    "port": 8000,
    "host": "0.0.0.0",
    "llamacpp": {
      "ctx_size": 32768,
      "n_gpu_layers": 999,
      "flash_attn": "on",
      "jinja": true,
      "fit": "on",
      "parallel": 1,
      "threads": -1,
      "cache_type_k": "f16",
      "cache_type_v": "f16",
      "spec_decoding": true,
      "spec_type": "draft-dflash",
      "spec_draft_n_max": 15,
      "extra_args": ""
    },
    "vllm": {
      "max_model_len": 32768,
      "gpu_memory_utilization": 0.92,
      "tensor_parallel_size": "auto",
      "enable_thinking": true,
      "enable_auto_tool_choice": true,
      "served_model_name": "laguna",
      "speculative_dflash": false,
      "num_speculative_tokens": 7,
      "extra_args": ""
    }
  }
}
'''


# run
./laguna_manager.sh start

# return start
'''
== Montando servidor ==
  Modelo  : laguna-s-2.1
  Backend : llamacpp
  GPUs    : 1 (cuda, 44 GB)
  Puerto  : 8000
Comando:
  /root/.laguna_manager/llama.cpp/build/bin/llama-server   -m   /root/.laguna_manager/models/laguna-s-2.1-GGUF/Laguna-S-2.1-UD-IQ1_M.gguf   --host   0.0.0.0   --port   8000   --ctx-size   32768   -ngl   999   -fa   on   --parallel   1   --jinja   --fit   on   --cache-type-k   f16   --cache-type-v   f16 

Servidor lanzado (PID 415). Logs: ./laguna_manager.sh logs -f
Esperando a que el servidor esté listo...
✔ Servidor listo en http://0.0.0.0:8000/v1  (OpenAI-compatible)
'''

# restart
./laguna_manager.sh restart

# return restart
'''
Deteniendo servidor (PID 866)...
Servidor detenido.
== Montando servidor ==
  Modelo  : laguna-s-2.1
  Backend : llamacpp
  GPUs    : 1 (cuda, 44 GB)
  Puerto  : 8000
Comando:
  /root/.laguna_manager/llama.cpp/build/bin/llama-server   -m   /root/.laguna_manager/models/laguna-s-2.1-GGUF/Laguna-S-2.1-UD-IQ1_M.gguf   --host   0.0.0.0   --port   8000   --ctx-size   32768   -ngl   999   -fa   on   --parallel   1   --jinja   --fit   on   --cache-type-k   f16   --cache-type-v   f16   -md   /root/.laguna_manager/models/laguna-s-2.1-DFlash/laguna-s-2.1-DFlash-BF16.gguf   --spec-type   draft-dflash   --spec-draft-n-max   15 

Servidor lanzado (PID 1783). Logs: ./laguna_manager.sh logs -f
Esperando a que el servidor esté listo...
✔ Servidor listo en http://0.0.0.0:8000/v1  (OpenAI-compatible)
'''



'''
./laguna_manager.sh gpu        # detecta GPUs (nvidia-smi/rocm-smi), VRAM total, RAM, disco
./laguna_manager.sh install    # auto-elige backend e instala lo que falte
./laguna_manager.sh download   # menú interactivo: modelo → backend → quant
./laguna_manager.sh start      # monta el servidor con tus hiperparámetros
./laguna_manager.sh stop       # lo detiene (con limpieza de workers de vLLM)
./laguna_manager.sh status     # PID, URL, modelos servidos, uso de VRAM
./laguna_manager.sh logs -f    # sigue los logs en vivo
'''