smollm2-135m-instruct-p150

smollm2-135m-instruct-p150 on Tenstorrent p150

Derived from HuggingFaceTB/SmolLM2-135M-Instruct by the Hugging Face SmolLM team, unmodified. See the original model card for training and evaluation details.

SmolLM2-135M-Instruct, a 135M-parameter Llama-architecture decoder-only transformer, served with vLLM.

Runs on Tenstorrent p150 (mesh P150) — 8,192-token context, up to 32 concurrent sequences.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Intended use

Smoke-testing and demonstrating tt-model packaging and serving: short English chat and instruction-following requests.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150

Quickstart

tt serve stisiTT/smollm2-135m-instruct-p150

tt serve downloads the Docker image and the HuggingFaceTB/SmolLM2-135M-Instruct weights (into your HF cache; they are not in the image), then starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve stisiTT/smollm2-135m-instruct-p150.

Capabilities

The server speaks the OpenAI API at http://127.0.0.1:20000/v1 (or whichever port your serve command reported; chat completions, completions, and /v1/models). Pass "model": "HuggingFaceTB/SmolLM2-135M-Instruct" — the weights id, not this package's name.

Expected performance

Accuracy against the reference model (HuggingFaceTB/SmolLM2-135M-Instruct in float32 on CPU):

IFEval scores are the mean of prompt- and instruction-level accuracy.

Medians of one benchmark run per row (4 to 256 requests each, random prompts), measured on 2026-10-07 on one p150 with the tt-inference-server benchmark workflow, on a warm server, against this package's image. Prefill tok/s/u is ISL divided by TTFT, so above concurrency 1 it includes queueing time.

Limitations

• Not for production or factual use.
• Greedy output diverges from the reference model's within the first few tokens (83% top-1 agreement).
• Contexts above 4,224 tokens are declared (up to 8,192) but not benchmarked.
• No tool-calling or reasoning parser is configured.
• Tested on one p150 only.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/stisiTT/smollm2-135m-instruct-p150/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: [email protected].

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image: