Downloads · 30 days
0
ndaly/Nemotron-3.5-Lightning-30B-A3B-NVFP4-tt-p300x2-vllm026
Nemotron-3.5-Lightning-30B-A3B-NVFP4-tt-p300x2-vllm026 is a text generation model from ndaly. Use it when you need the model to write or continue text. The card lists the license as other.
Derived from nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 by NVIDIA, unmodified. See the original model card for training and evaluation details.
Downloads · 30 days
0
Access
Public
Updated Oct 21, 2026
Repo size
1.9 GB
Likes
0
Public
Click a slice to open those files.
Other1.9 GB · 100%
From the Hugging Face model README
Derived from nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 by NVIDIA, unmodified. See the original model card for training and evaluation details.
NVIDIA Nemotron 3.5 Lightning 30B-A3B (the NVFP4 checkpoint), a 30B-parameter hybrid Mamba-2 / attention mixture-of-experts reasoning model with about 3B active parameters per token and a 1,048,576-token context, served with vLLM 0.26.0. Text only; thinking is on by default and comes back separately from the answer. Superseded by ndaly/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4, which fixes the serving failures listed under Limitations.
Runs on Tenstorrent p300x2 (mesh P300x2) — 1,048,576-token context, up to 32 concurrent sequences.
Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).
| Hardware | p300x2 |
| Context | 1,048,576 tokens |
| License | OpenMDW-1.1 |
Reasoning chat, tool calling and long-context (up to 1M tokens) text serving through the OpenAI-compatible API, for one user at a time.
tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)tt serve ndaly/Nemotron-3.5-Lightning-30B-A3B-NVFP4-tt-p300x2-vllm026
tt serve downloads the Docker image and the nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 at 6dbbd757ea75a8ece6e0702872e3ae53f9987728 weights (into your HF cache; they are not in the image), then starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.
Without tt-cli: tt-model serve ndaly/Nemotron-3.5-Lightning-30B-A3B-NVFP4-tt-p300x2-vllm026.
The first boot also converts the NVFP4 weights to the device layout; with kernel compilation it takes
about 12 minutes, and later boots about 4 minutes from the cache. After any non-clean stop, run
tt-smi -r all before serving again.
The server speaks the OpenAI API at http://127.0.0.1:20000/v1 (or whichever port your serve command reported; chat completions, completions, and /v1/models). Pass "model": "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4" — the weights id, not this package's name.
Tool calling is enabled (--tool-call-parser qwen3_coder): a request that passes tools comes back with tool_calls and finish_reason: tool_calls.
Reasoning output is separated (--reasoning_parser nemotron_v3): the thinking text arrives in reasoning_content, apart from content.
No accuracy eval against the reference model was recorded for this package.
| profile | ISL | OSL | concurrency | N | TTFT (ms) | prefill tok/s/u | decode tok/s/u | E2EL (s) |
|---|---|---|---|---|---|---|---|---|
| default | short prompt | — | 1 | — | 110 | — | 24 | — |
Measured on p300x2 through the served endpoint on 2026-09-09; decode is 42 ms per token. Prompt and output lengths and the number of runs were not recorded.
/health while serving nothing; stop it, run
tt-smi -r all, and serve again.chat_template_kwargs: {"enable_thinking": false} the parser finds no </think> and returns
the answer in reasoning_content with empty content.max_tokens is reported as empty content with
finish_reason: length; raise max_tokens or ask for shorter reasoning.Weights: NVIDIA Open Model License (OpenMDW-1.1, see the weights repo). Port and serving code in
code/: Apache-2.0.
Questions or problems with this package: open a discussion at https://huggingface.co/ndaly/Nemotron-3.5-Lightning-30B-A3B-NVFP4-tt-p300x2-vllm026/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: [email protected].
The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:
| component | built from |
|---|---|
| tt-metal | a local checkout — commit not published (dirty tree — the image includes uncommitted changes) |
| vLLM | v0.26.0 |
| vllm-tt-plugin | a local checkout — commit not published |
code/ digest | 5787b8dfce9d7e38 (sha256, first 16 hex digits) |
| built | 2026-09-11T17:47:54+00:00 by tt-model 0.1.0 |