laguna-s-2.1-v51

laguna-s-2.1

Laguna-S-2.1 — 118B hybrid-attention MoE (~4B active/token) served on 4x Tenstorrent Blackhole (p150x4, TP=4+EP=4, FABRIC_1D) via vLLM with the tenstorrent/vllm-tt-plugin.

Runs on p150x4 (mesh P150x4) — 8,192-token context, up to 1 concurrent sequences.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  mando2222/laguna-s-2.1-v51 --with-weights
tt-model serve mando2222/laguna-s-2.1-v51

pull --with-weights downloads the Docker image and the poolside/Laguna-S-2.1 weights (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Context length

max_model_len is 8192, well below what this model's weights would allow. That is a serving-stack limit, not a model limit: the TT vLLM plugin allocates a UNIFORM full-attention KV cache per layer, so Laguna's 36 sliding-window-512 layers are costed as if they were full-context. Byte-math on the real attention pattern would allow ~1M tokens; until the plugin can allocate per-layer sliding KV, 8192 is what fits.

Use with your client

Point any OpenAI-compatible client at http://127.0.0.1:20000 once the server is up (or the port tt-model serve printed). Served model names: poolside/Laguna-S-2.1, laguna-s-2.1.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image: