olmo-3.1-32b-instruct-blackhole

olmo-3.1-32b-instruct on Tenstorrent p150 and p300 and p300x2

Derived from allenai/Olmo-3.1-32B-Instruct by AI2, unmodified. See the original model card for training and evaluation details.

Olmo 3.1 32B Instruct, a fully open 32B dense decoder with hybrid sliding-window / full attention and 65k context, served with vLLM.

Runs on Tenstorrent p150 or p300 or p300x2 — see the serving profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Intended use

General chat and tool-calling assistance through an OpenAI-compatible API.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150 or p300 or p300x2

Quickstart

tt serve jashansinghTT/olmo-3.1-32b-instruct-blackhole

tt serve downloads the Docker image and the allenai/Olmo-3.1-32B-Instruct weights at ac0587e4a7744a551c059d8cd17ba220bc940dae (into your HF cache; they are not in the image), then starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve jashansinghTT/olmo-3.1-32b-instruct-blackhole.

Serving profiles

Select a different serving profile using --profile.

Capabilities

The server speaks the OpenAI API at http://127.0.0.1:20000/v1 (or whichever port your serve command reported; chat completions, completions, and /v1/models). Pass "model": "allenai/Olmo-3.1-32B-Instruct" — the weights id, not this package's name.

Tool calling is enabled (--tool-call-parser olmo3_tt): a request that passes tools comes back with tool_calls and finish_reason: tool_calls.

Expected performance

IFEval with lm-evaluation-harness 0.4.10 through the chat endpoint (one sample per prompt, seed 42); reference is the upstream model card. Speed from the tt-model latency sweep on a QuietBox 2 (four Blackhole p300c chips), September 2026; N is the number of requests.

Limitations

• On p150, about 1k-token prompts at 32 concurrent users, or 4k-token prompts at 8 or more, stop the engine returning tokens. Keep long-prompt concurrency low on that profile, or use p300/p300x2.
• Prompts above 16k tokens were measured at one user only; at 16k tokens and 32 users on p300, a request takes about 317 s end to end.
• Only IFEval was measured for accuracy; math, reasoning and code benchmarks were not.
• Not for image or audio input (text only), or for prompts above 16,384 tokens on p150 or 65,536 tokens on the other profiles.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/jashansinghTT/olmo-3.1-32b-instruct-blackhole/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: [email protected].

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image: