Downloads · 30 days
0
mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv
qwen3.8-27b-dflash2-vision-p300x2-q4kv is a machine learning model from mando2222. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Unified Qwen/Qwen3.8-27B serving on a Tenstorrent P300x2 (4 Blackhole chips, 4-way tensor parallel), release unified-v1.1-driftfix (fixes output corruption under concurrent requests, see below): Q4 KV cache, up to eig…
Downloads · 30 days
0
Access
Public
Updated Oct 13, 2026
Repo size
7.7 GB
Likes
3
Public
Click a slice to open those files.
Other1.9 GB · 99%
From the Hugging Face model README
Unified Qwen/Qwen3.8-27B serving on a Tenstorrent P300x2 (4 Blackhole chips, 4-way tensor parallel), release unified-v1.1-driftfix (fixes output corruption under concurrent requests, see below): Q4 KV cache, up to eight mixed text/image requests, up to four images per request, automatic lossless DFlash2 acceleration for compatible greedy requests, and an ordinary-decode fallback for sampling, logprobs, penalties, constraints, and structured output. The immutable vision-baseline-v1 tag preserves the pre-unification implementation for bisecting.
Runs on p300x2 (mesh P150x4) — 262,144-token context, up to 8 concurrent sequences.
Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).
tt-model pull mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv --with-weights
tt-model serve mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv
pull --with-weights downloads the Docker image and automatically downloads both required checkpoints: Qwen/Qwen3.8-27B at 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, incoai/Qwen3.8-27B-DFlash2 at dedf8df68adfb1afeaf7b7480c0a0243108177b4 (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.
OpenAI-compatible API on port 20000, model id Qwen/Qwen3.8-27B:
curl -s http://localhost:20000/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"model": "Qwen/Qwen3.8-27B", "messages": [{"role": "user", "content": "Write a Python function that merges two sorted lists."}], "max_tokens": 512}'
unified is the only public profile and the default. It accepts up to eight concurrent text/image requests and up to four images per request. Eligible greedy requests use DFlash automatically; requests with supported sampling or output controls use the plain lane without silently dropping parameters.
tt-model serve mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv
tt-model profiles mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv
The endpoint is synchronous and prefix caching is disabled. Features unsupported by both execution lanes return a descriptive 4xx response.
--generation-config auto with vllm, which drops the model's generation_config.json defaults. This bundle restores them: temperature 1.0, top_k 20, top_p 0.95. Without them, requests that omit top_k/top_p sample from the full vocabulary and occasionally emit stray Chinese/Cyrillic tokens.reasoning_effort: xhigh. This bundle defaults to medium. Override it per request with "chat_template_kwargs": {"reasoning_effort": "xhigh" | "medium" | "low"}.temperature: 0 take the DFlash lane at every prompt length (the benchmark numbers below). Sampled requests take the plain lane (about 20-30 tok/s for one user). OpenCode only sends a configured temperature for a custom-provider model whose entry declares "temperature": true.unified-v1 could corrupt in-flight requests when another request joined the decode batch. Requests emitted duplicated or garbled tokens; agents saw this as malformed tool calls. A deterministic exact-copy test with 6 staggered clients corrupted about 32% of outputs. v1.1 fixes both causes in the qwen36 GatedDeltaNet state handling:
gdn_decode_step stores user b at tile row 2c + (b & 1), so a request that moved between an even and an odd slot decoded on a garbage conv window.v1.1 also sizes decode buckets from the highest active row, so live requests above a gap are no longer skipped.
| v1.1 validation | Result |
|---|---|
| Staggered exact-copy test, 1/2/6/8 clients; synchronized rounds; single victim + joiner | 0 corrupted (0/42, 0/73, 0/124, 0/145; 0/72; exact) |
| DFlash concurrent parity, 4 and 8 users; unified API gate; reasoning gate | PASS; PASS; 18/18 |
| qwen36 device tests (spec-join exactness, DFlash2 serving losslessness, decode bucketing, GDN TP, new remap tests) | all pass |
Performance vs unified-v1, interleaved boots | within noise (<1%): DFlash decode 57.9 / 51.6 tok/s short / long, prefill 6,583 tok/s at 7K, plain and DFlash at 1/4/8 users, prefill 2.7K-92K |
Final unified validation on four Blackhole p300c chips, firmware 19.15.0.0:
| Gate | Result |
|---|---|
| Functional / tools / reasoning | 10/10; 4/4; xhigh/medium/low 18/18 |
| Decode / prefill | 54.48 tok/s short; 80.59 tok/s long; 6,679.3 prefill tok/s |
| Context / concurrency | 260,340 tokens; 4 agents in 8.4 s; 8 in 15.9 s |
| Vision | DFlash 49.472 tok/s; unified plain 31.503 tok/s (1.570x); plain is +2.56% vs the 30.717 tok/s standalone reference |
| Exact greedy parity | 371 image tokens, 384 text tokens, and all 4/8 concurrent outputs |
| Q4 shared pool | 8 x 129,298-token prompts passed at 98.4% occupancy |
Historical pre-unification measurements from the immutable vision-baseline-v1 implementation follow. t/s = aggregate output tokens per second, (t/s/u) = per user; plain = the same box without speculation (changh95/qwen3.8-27b-p300x2).
batch8-dflash2, t/s (t/s/u), 1, 2, 4, 8 concurrent users| ISL | OSL | 1 user | 2 users | 4 users | 8 users | plain 1 user |
|---|---|---|---|---|---|---|
| 128 | 128 | 56.2 (56.2) | 100.7 (53.1) | 182.2 (46.2) | 271.5 (34.9) | 32.3 |
| 1,024 | 128 | 53.3 (53.3) | 98.4 (49.9) | 158.8 (42.3) | 243.6 (31.3) | 31.3 |
| 2,048 | 128 | 51.6 (51.7) | 86.5 (44.4) | 131.8 (34.8) | 194.3 (24.9) | 30.7 |
| 4,096 | 128 | 45.8 (45.8) | 76.6 (38.8) | 103.5 (26.9) | 135.0 (17.1) | 28.8 |
| 8,192 | 128 | 35.1 (35.1) | 52.8 (26.5) | 71.8 (18.1) | 80.4 (10.4) | 25.9 |
| 16,384 | 128 | 26.0 (26.0) | 33.8 (17.1) | 40.9 (10.3) | 45.2 (5.7) | 21.2 |
| 32,768 | 128 | 16.1 (16.1) | 19.8 (10.2) | 22.0 (5.6) | 23.3 (2.9) | 15.1 |
| 65,536 | 128 | 9.7 (9.7) | 10.3 (5.2) | 10.7 (2.7) | 9.8 (1.2) | 9.2 |
| 131,072 | 128 | 4.4 (4.4) | 4.6 (2.3) | 4.7 (1.2) | – | 4.6 |
| 128 | 1024 | 51.4 (51.4) | 86.7 (43.9) | 168.1 (49.1) | 297.3 (41.8) | 32.6 |
| 8,192 | 1024 | 57.6 (57.6) | 63.3 (33.3) | 125.5 (36.3) | 198.5 (30.1) | 31.1 |
| 10,000 | 1024 | 58.5 (58.5) | 64.5 (33.9) | 102.1 (33.5) | 185.8 (26.3) | 30.9 |
single-user-dflash2, one user| ISL | OSL | TTFT ms | TPOT ms | t/s/u | plain t/s/u |
|---|---|---|---|---|---|
| 128 | 128 | 134 | 15.0 | 62.7 | 32.3 |
| 1,024 | 128 | 183 | 14.2 | 64.3 | 31.3 |
| 2,048 | 128 | 304 | 16.0 | 54.9 | 30.7 |
| 4,096 | 128 | 594 | 17.5 | 45.4 | 28.8 |
| 8,192 | 128 | 1,187 | 16.1 | 39.6 | 25.9 |
| 16,384 | 128 | 2,460 | 17.0 | 27.7 | 21.2 |
| 32,768 | 128 | 5,073 | 21.2 | 16.5 | 15.1 |
| 65,536 | 128 | 11,163 | 20.8 | 9.3 | 9.2 |
| 131,072 | 128 | 26,055 | 12.1 | 4.6 | 4.6 |
| 128 | 1024 | 135 | 12.1 | 82.1 | 32.6 |
| 8,192 | 1024 | 1,197 | 17.9 | 52.5 | 31.1 |
| 10,000 | 1024 | 1,515 | 14.3 | 63.6 | 30.9 |
coding, 80 prompts; greedy; answers end at EOS), t/s (tok/s per user); speed-up per user| thinking | users | batch8-dflash2 | single-user-dflash2 | plain | speed-up |
|---|---|---|---|---|---|
| off | 1 | 84 (111) | 95 (119) | 35 (36) | 3.3× |
| off | 2 | 152 (107) | – | – | – |
| off | 4 | 247 (102) | – | – | – |
| off | 8 | 233 (60) | – | 111 (23) | 2.6× |
| on | 1 | 73 (77) | 80 (83) | – | – |
| on | 2 | 143 (76) | – | – | – |
| on | 4 | 268 (73) | – | – | – |
| on | 8 | 397 (55) | – | 218 (29) | 1.9× |
t/s = aggregate output tokens per second over the run, which includes prefill and the gaps between a user's requests. thinking off passes "chat_template_kwargs": {"enable_thinking": false}; thinking on is the served default (the harness's prompts), with answers capped at 4,096 tokens. Speed-up = per-user speed of the faster speculative profile over plain decode at the same user count.The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:
| component | built from |
|---|---|
| tt-metal | a local checkout — commit not published |
| vLLM | v0.26.0 |
| vllm-tt-plugin | a local checkout — commit not published |
code/ digest | 43a221155529e768 (sha256, first 16 hex digits) |
| built | 2026-10-13T04:35:19+00:00 by tt-model 0.1.0 |