sam3-p150

sam3-p150

Meta SAM3 (Segment Anything with Concepts) port on one Tenstorrent Blackhole p150a. Weights: facebook/sam3 · Paper: arXiv:2511.16719 · Upstream code: facebookresearch/sam3 · Port: changh95/tt-sam3

Runs on p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart (Python)

Prerequisite: a Python environment with tt-metal's ttnn, built at tt-metal 8b98410e730 with patches/tt-metal-eth-dispatch.patch applied. The facebook/sam3 weights are gated. Request access on the Hub and run hf auth login one time.

hf download changh95/sam3-p150 --exclude "image/*" --local-dir sam3-p150 && cd sam3-p150
pip install -e code/      # into a Python env that already has tt-metal's ttnn
pip install -e "code/[server,test]"     # also the HTTP server (fastapi, uvicorn, ...) and the tests (pytest, httpx)

Run the Python example from the model repo root. The demo image path media/source_1.png is relative to the repo root. You can also give an absolute path.

from tt_sam3 import Sam3

with Sam3.from_pretrained(device_id=0) as model:     # loads facebook/sam3 from the HF cache, opens the chip, warms up
    result = model("media/source_1.png", "car")      # path, PIL.Image, numpy or torch image; str or list of prompts

print(result)          # Sam3Result(prompt='car', num_instances=3, scores=[0.973, 0.817, 0.797], ...)
result.masks           # (N, H, W) bool masks at the original image size
result.boxes           # (N, 4) float32 [x0, y0, x1, y1] in image pixels
result.scores          # (N,) float32, high to low

• from_pretrained() opens the chip, loads the weights and captures the device traces. The first run compiles the kernels and can take some minutes.
• Write model = Sam3.from_pretrained(warmup_variants={"prompts": ["car", "person"]}) to prepare the model and the text of your prompts. The first call is then as fast as later calls. A prompt that you did not list adds about 80 to 175 ms to the first call that uses it.
• Use with ... as model: or call model.close() to release the chip.
• Give a list of prompts to segment one image with many prompts. The image part runs one time, and each prompt adds about 20 ms.
• The model keeps the last image on the device. When the same image comes again, the call takes about 21 ms.
• model.predict_many(images, prompts) decodes the next image on a host thread while the chip runs the current image.
• model(...) gives the same results as the HTTP POST /predict, without JSON rounding.

Full reference: code/PYTHON.md. Example with an overlay image: examples/quickstart.py.

Serving (HTTP)

tt-model pull  changh95/sam3-p150 --with-weights
tt-model serve changh95/sam3-p150        # or, with tt-cli: tt serve changh95/sam3-p150
printf '{"image":"%s","prompts":["car"]}' "$(base64 -w0 media/source_1.png)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
tt model stop changh95/sam3-p150

• Weights facebook/sam3 go to your HF cache; the image does not contain them.
• Serves on port 20000 (or the next free port); ready when the log says Application startup complete.
• POST /predict: image (base64 PNG/JPEG), prompts (1–4 noun phrases); optional threshold (0.5), max_instances (50), mask_format (png | rle | none).
• POST /predict_raw: the same results with a binary body. Send PNG/JPEG bytes, or raw RGB24 pixels with Content-Type: application/x-rgb24 and the X-Width and X-Height headers. Give prompts and the options as query parameters. This route is in code/ and is not in the container image yet.
• GET /health, GET /info.

{"width": 1920, "height": 1080, "input_size": 1008, "threshold": 0.5, "mask_format": "png",
 "results": [{"prompt": "car", "presence": 0.995, "num_instances": 3, "instances": [
    {"score": 0.9727, "box_xyxy": [545.8, 447.9, 1159.1, 854.4], "mask_area": 175127, "mask_png": "..."}, "..."]}],
 "timing_ms": {"decode": "...", "preprocess": "...", "forward": "...", "postprocess": "...", "total": "..."}}

• One results entry per prompt; instances are sorted by score, box_xyxy is [x0, y0, x1, y1] in original image pixels.
• mask_png is a base64 8-bit 0/255 PNG at the original H×W; rle gives COCO uncompressed column-major {size, counts}, none omits masks.

Demo

Synthetic street scene, prompt "car".

Demo & Performances

Warm, batch 1, demo image (1920×1080 PNG) + "car", served /predict, 2 runs of 20 requests (median per run). All rows use the p150 configuration: dispatch on ETH cores, 1 command queue (CQ), 12×10 compute grid.

The measurement hardware is a Blackhole chip with a 12×10 compute grid and dispatch on ETH cores (patches/tt-metal-eth-dispatch.patch), with 1 command queue. This is the configuration of a single p150. On 2026-10-05, the developer and an independent verifier measured the served rows and the HTTP rows again in this configuration (x1 PCIe link, shared host). Each range shows the two runs. The trace rows and the forward_image row are from the 2026-10-03 verification (x8 PCIe link). Accuracy against the torch CPU fp32 reference (demo image, "car"): top-mask IoU288 0.9986, 3/3 instances, top score 0.9731 (reference 0.9716), all-200 mask PCC 0.9806 (gate 0.98). Details: VERIFICATION_2026-10-03.md.

Before 2026-10-05, this card showed a served total of 96.8 ms (1 prompt) and 137 ms (3 prompts). The server measured these numbers with worker (Tensix) dispatch. On the measurement chip, worker dispatch also gives 12×10, but on a p150 it gives only 11×10. ETH dispatch gives the same numbers within about 2 ms, and the outputs are bit-identical. Details: VERIFICATION_2026-10-03.md.

The 2026-10-04 code/ adds the Python API, the image cache and POST /predict_raw. These changes do not change the device programs or the output bits. Thus, the device numbers and the accuracy stay the same. The Python row uses chip 14, SAM3_NUM_THREADS=8 and 3 interleaved rounds of 10 calls. With a file path as input, the call takes 97.0 ms median because it includes the PNG decode. Details: VERIFICATION_2026-10-03.md.

RTX 5090 reference measurements (2026-09-14) are unchanged. They use the port's torch reference in eager PyTorch 2.11, batch 1, with H2D/D2H included, plus the original PIL host decode, preprocess and postprocess. The current build values are the means of the two 2026-10-05 runs. Full table: GPU_COMPARISON.md.

Most of the end-to-end lead comes from the host path. This build spends about 14 ms on decode and postprocess. The GPU loop spends 47.2 ms on the same steps with PIL. On the model forward alone, bf16 and fp16 GPU are about 1.3× faster.

Caveats

• Does not scale to multiple p150a in a mesh configuration. The current build uses a 12x10 compute grid of Tensix cores. To get this grid, the dispatch functions move from 10 Tensix cores to ETH cores (patches/tt-metal-eth-dispatch.patch). Thus, this build assumes that you do not need chip-to-chip ethernet communication.
• The newer code/ opens the chip with ETH dispatch and 1 CQ by default (SAM3_DISPATCH=auto, also dispatch="auto" in the Python API). SAM3_DISPATCH=worker is an opt-in for Galaxy chips only. It gives an 11×10 grid on a p150. Without the ETH-dispatch patch, auto uses worker dispatch and logs a warning. The container image was built before this change, and its server uses worker dispatch until the image is rebuilt.
• Every image is squashed to 1008×1008. One image per request, batch 1. Text prompts only (no box or point prompts). Each additional prompt adds one prompt trace (about 20 ms served).
• facebook/sam3 is gated under the Meta SAM License: request access and hf auth login on the host before serve (3.45 GB sam3.pt).
• The whole image graph runs on the device as two metal traces. The text encoder runs on the host CPU, and its output is cached per prompt.
• The all-200 demo mask PCC is 0.9806 (previous release: 0.984). The margin to the 0.98 gate is small. The device neck, decoder and seg head change numerics, and so do some kernel changes (GELU, GroupNorm, SDPA chunk sizes). OPT_REPORT.md lists each change.
• Not an OpenAI-compatible API; GET /v1/models is a stub so the tt-model ready card does not 404.
• p150a power was not measured, so no efficiency comparison is made.

Licensing

• Weights: facebook/sam3, Meta SAM License (gated, not OSI).
• Port and serving code (code/): from changh95/tt-sam3, distributed under the same SAM License terms; code/tt_sam3/assets/bpe_simple_vocab_16e6.txt.gz is the MIT-licensed CLIP BPE vocabulary vendored from the upstream sam3 repo.

Provenance

These are the exact sources the container image was built from. code/ has since been updated (2026-10-04 build: optimized device path, image cache, /predict_raw and the Python API; 2026-10-05: ETH dispatch and 1 CQ are the default for the server and the harnesses; see OPT_REPORT.md and code/PYTHON.md) and is newer than the image. tt-model serve runs the image's code until the image is rebuilt. tt-model.yaml and SERVING.md still describe the image: