qwen3.6-a3b-blackhole

qwen3.6-a3b-blackhole

Qwen3.6-35B-A3B (hybrid Gated-DeltaNet + full attention, 256-expert MoE, 8 active) served on a single Tenstorrent Blackhole P150a through vLLM. Multi-turn prefix reuse keeps time-to-first-token roughly flat as a conversation grows, instead of re-prefilling the whole transcript each turn.

Runs on p150 (mesh P150) — 262,144-token context, up to 1 concurrent sequences.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  Adartras/qwen3.6-a3b-blackhole --with-weights
tt-model serve Adartras/qwen3.6-a3b-blackhole

pull --with-weights downloads the Docker image and the Qwen/Qwen3.6-35B-A3B weights at 995ad96eacd98c81ed38be0c5b274b04031597b0 (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Serve

tt-model serve Adartras/qwen3.6-a3b-blackhole

Then point any OpenAI-compatible client at the port it prints (tt-model serve picks one from 20000 upward; a bare docker run binds 8000).

What to expect on one P150a

• decode ~37 tok/s, prefill ~600-880 tok/s
• multi-turn TTFT ~0.3 s steady on a growing chat, 96-99% of each prompt reused
• up to 4 conversations stay warm at once, so a client that interleaves prompt streams (a chat UI's auto-title requests alongside your conversation) keeps its prefix. Slots age out least-recently-used; QWEN36_CONV_SLOTS=1 restores single-conversation behaviour
• 256K context ceiling (QWEN36_MAX_SEQ), bf4 experts, bf8 KV. KV is paged, one pool shared by the conversation slots, so this costs the same memory as a single flat cache

Reasoning ("thinking") blocks are parsed out via --reasoning-parser qwen3, and tool calls via --tool-call-parser qwen3_coder.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image: