Brazley-Nemotron-3.5-Lightning-Omni

Brazley Nemotron 3.5 Lightning Omni

Unofficial model created by Nik Brazley — not an NVIDIA release.

Omni's vision and audio towers on Lightning's text tower: a multimodal variant of NVIDIA Nemotron 3.5 Lightning that NVIDIA has not published.

Not an NVIDIA release

This is an independent derivative model. It is not an official NVIDIA product, it is not endorsed by, affiliated with, sponsored by, or supported by NVIDIA Corporation, and NVIDIA has not reviewed, validated, or approved it.

NVIDIA has not published a multimodal variant of Nemotron 3.5 Lightning. This model was assembled by Nik Brazley by combining two separately published NVIDIA checkpoints, and the "Brazley" in its name identifies that source. The behaviour of this model is the responsibility of that merge — not of NVIDIA or of either source model.

"NVIDIA", "Nemotron", and "DGX Spark" are trademarks of NVIDIA Corporation, used here solely to identify the origin of the source models in accordance with Section 4 of the NVIDIA Open Model Agreement. No trademark rights are claimed and no affiliation is implied.

For NVIDIA's official models, see nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 and nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4.

At a Glance

Quick Start Guide

Model Parameters

Send frequency_penalty: 0.1 on every request, in every mode.

Model Overview

Description:

Brazley Nemotron 3.5 Lightning Omni is a multimodal model built from two NVIDIA checkpoints. Its architecture, vision tower and audio tower come from NVIDIA Nemotron 3 Nano Omni V2; its text tower is NVIDIA Nemotron 3.5 Lightning, transplanted whole. NVIDIA publishes Lightning as a text-only model and has not released a multimodal variant of it. Nano Omni carries modality towers that Lightning does not, and the two checkpoints share enough structure that the text tower can be exchanged at the seam between them — this model is the result of trying it.

No training was performed. Zero gradient steps: no fine-tuning, no distillation, no adapter, no calibration pass over data. The model is a module transplant at the architectural seams, followed by quantization. Every weight is NVIDIA's, copied byte-for-byte from the source safetensors.

Because there were no gradient steps, building it has an inference-shaped compute profile rather than a training-shaped one — load, transplant, quantize, validate. That is what put it within reach here: one person and a set of AI agents, working on an M5 MacBook Pro driving a cluster of NVIDIA DGX Spark boxes, Thunderbolt from the Mac to the Sparks and the Spark fabric between the Sparks. DGX Spark is listed above as the minimum GPU for running this model.

Everything needed to run it is below — the container, the exact launch command, the sampling parameters, the full quantization composition, and the numbers it was measured at. It is available for commercial use under the terms of both governing licenses. Go and enjoy it.

License/Terms of Use

Governing Terms: use of this model is governed by both the NVIDIA Open Model Agreement and the OpenMDW License Agreement v1.1, which apply to different parts of the model. Both must be complied with. Full texts ship in this repository; see License for the component breakdown and downstream obligations.

Third-party code loaded at runtime: this model's config.json references nvidia/C-RADIOv2-H via auto_map for the vision-tower implementation. Loading with trust_remote_code=True fetches that code directly from NVIDIA, under the NVIDIA Open Model License Agreement (June 14, 2024) — a separate and older agreement from the one governing this model. It is not redistributed here.

Deployment Geography:

Global

Use Case:

Users who want Nemotron 3.5 Lightning's text behaviour with multimodal input:

• Document and image understanding where the downstream reasoning is the hard part
• Speech transcription (ASR) and audio Q&A
• Video question answering and temporal description
• Speculative-decoding deployments on a single DGX Spark

Release Date:

Hugging Face 08/20/2026 via:

• brazley/Brazley-Nemotron-3.5-Lightning-Omni
• brazley/Brazley-Nemotron-3.5-Lightning-Omni-Drafter

Model Architecture:

Architecture Type: Mamba2-Transformer Hybrid Mixture of Experts (MoE), multimodal

Resolved architecture: NemotronH_Nano_VL_V2

Network Architecture:

• Nemotron 3.5 Lightning (30B A3B) — the text tower, transplanted unmodified. 52 layers, hidden 2688, vocab 131072, 32 heads / 2 KV, head_dim 128.
• C-RADIO v2-H — vision encoder for image and video frames, from the Nano Omni checkpoint.
• Parakeet — speech encoder for audio inputs, from the Nano Omni checkpoint.
• mlp1 and sound_projection — modality projectors, from the Nano Omni checkpoint.

Number of model parameters: 3.44 × 10^10 (34B total, ~3B active per token)

Input(s):

Input Type(s): Video, Audio, Image, Text

Input Format(s):

• Video: mp4
• Audio: wav, mp3. Send raw base64 in input_audio.data — not a data: URL (see Notes)
• Image: RGB (jpeg, png)
• Text: String

Input Parameters:

• Video: Three-Dimensional (3D)
• Audio: One-Dimensional (1D)
• Image: Two-Dimensional (2D)
• Text: One-Dimensional (1D)

Other Properties Related to Input:

• Maximum context length up to 262,144 tokens
• Language support: English only

Output(s)

Output Type(s): Text

Output Format(s):

• Text: String

Output Parameters:

• Text: One-Dimensional (1D)

Other Properties Related to Output:

• Maximum context length up to 262,144 tokens
• Supports reasoning output with chain-of-thought via enable_thinking
• Tool calling requires launching with a tool parser — the reference invocation below does not include one, and a request carrying tools returns HTTP 400 until it does

Software Integration:

Runtime Engine(s):

• vLLM 0.27.1

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Blackwell (DGX Spark / GB10)

Preferred/Supported Operating System(s):

• Linux (aarch64 / ARM64)

Inference Runtimes:

• vLLM — verified
• SGLang — the required sampling parameters are all expressible, but this model has not been served on it

This AI model can be embedded as an Application Programming Interface (API) call into the software environment described above.

Model Version(s):

Brazley-Nemotron-3.5-Lightning-Omni

---

Download Model Weights

pip install -U "huggingface_hub[hf_xet]"

WEIGHTS=/path/to/Brazley-Nemotron-3.5-Lightning-Omni
DRAFTER=/path/to/Brazley-Nemotron-3.5-Lightning-Omni-Drafter

hf download brazley/Brazley-Nemotron-3.5-Lightning-Omni --local-dir "$WEIGHTS"
hf download brazley/Brazley-Nemotron-3.5-Lightning-Omni-Drafter --local-dir "$DRAFTER"

---

vLLM

Required version: vLLM 0.27.1.

Container

You will want to build your own image rather than use the upstream one. On a DGX Spark, vllm/vllm-openai:v0.27.1 cannot compile CPU kernels for ARMv9 — it surfaces as a multimodal processor error at startup, and disabling video just moves it to the next modality that compiles. Adding a compiler that knows ARMv9 fixes it, and it is a three-line Dockerfile:

FROM vllm/vllm-openai:v0.27.1
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y g++-12 gcc-12 \
 && rm -rf /var/lib/apt/lists/*
ENV CXX=g++-12 CC=gcc-12
RUN pip install --no-cache-dir librosa soundfile   # audio path

docker build -t omni-vllm:v0.27.1-gcc12-audio .

This applies to any torch/vLLM container on GB10 with gcc < 13, not only to this model.

General Invocation (1× DGX Spark)

docker run -d --name omni-lightning \
  --gpus all --shm-size 32g --network host --ipc=host \
  -v "${WEIGHTS}:/models/Brazley-Nemotron-3.5-Lightning-Omni:ro" \
  -v "${DRAFTER}:/models/Brazley-Nemotron-3.5-Lightning-Omni-Drafter:ro" \
  -v "${DRAFTER}/nano_omni_eagle3.py:/usr/local/lib/python3.12/dist-packages/nano_omni_eagle3.py:ro" \
  -v "${DRAFTER}/nano_omni_eagle3-0.1.dist-info:/usr/local/lib/python3.12/dist-packages/nano_omni_eagle3-0.1.dist-info:ro" \
  omni-vllm:v0.27.1-gcc12-audio \
  --model /models/Brazley-Nemotron-3.5-Lightning-Omni \
  --served-model-name omni-lightning-nvfp4 \
  --tensor-parallel-size 1 \
  --trust-remote-code \
  --gpu-memory-utilization 0.65 \
  --max-model-len 262144 \
  --reasoning-parser nemotron_v3 \
  --speculative-config '{"method":"dspark","model":"/models/Brazley-Nemotron-3.5-Lightning-Omni-Drafter","num_speculative_tokens":3}' \
  --port 8200

The two nano_omni_eagle3 mounts are required for speculative decoding. vLLM's DSpark path requires the target model to satisfy the SupportsEagle3 protocol, which the Omni multimodal wrapper does not declare — stock Nano Omni has no speculative-capable text tower, and this one does only because of the Lightning transplant. The shim declares it as a standard vllm.general_plugins entry point; no forked engine and no rebuilt image. Without the mounts the server still starts, serves, and reports healthy, and runs roughly 12 tok/s slower. Confirm the drafter loaded by checking the log for:

[nano-omni-eagle3] DRAFT WEIGHTS VERIFIED: 118 checkpoint tensors -> 92 draft
parameters (702.4M elements); 39/39 directly-comparable tensors match the
checkpoint EXACTLY, 0 mismatches.

To serve without speculative decoding, drop --speculative-config and both shim mounts.

Key Spark-Specific Flags

Verify the server is ready:

curl -sS http://localhost:8200/v1/models | python3 -m json.tool

---

API Client (OpenAI-compatible)

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8200/v1", api_key="")
MODEL = "omni-lightning-nvfp4"

Image Example

import base64

def image_to_data_url(path: str) -> str:
    with open(path, "rb") as f:
        b64 = base64.b64encode(f.read()).decode("utf-8")
    return f"data:image/jpeg;base64,{b64}"

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "Describe this image in detail."},
        {"type": "image_url", "image_url": {"url": image_to_data_url("example.jpeg")}},
    ]}],
    max_tokens=2048,
    temperature=0.2,
    frequency_penalty=0.1,
    extra_body={"top_k": 1, "chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)

Audio Example

import base64

with open("speech.wav", "rb") as f:
    audio_b64 = base64.b64encode(f.read()).decode("utf-8")   # RAW base64, no data: prefix

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": [
        {"type": "input_audio", "input_audio": {"data": audio_b64, "format": "wav"}},
        {"type": "text", "text": "Transcribe this audio."},
    ]}],
    max_tokens=2048,
    temperature=0.2,
    frequency_penalty=0.1,
    extra_body={"top_k": 1, "chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)

Text Example (curl)

curl -sS http://localhost:8200/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"omni-lightning-nvfp4",
       "messages":[{"role":"user","content":"Hello, what can you do?"}],
       "temperature":0.2,"top_k":1,"frequency_penalty":0.1,
       "chat_template_kwargs":{"enable_thinking":false}}' \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['choices'][0]['message']['content'])"

---

Reasoning Mode (enable_thinking)

To disable reasoning on a request, add to the JSON body:

"chat_template_kwargs": {"enable_thinking": false}

Read all three channels. With --reasoning-parser nemotron_v3, an OpenAI client that reads only .content gets nothing back from a request that returned HTTP 200 with finish_reason: stop and billed completion tokens. Accumulate content, reasoning_content and reasoning, and cross-check the total against usage.completion_tokens — that way "the model emitted nothing" and "we failed to capture it" cannot produce identical output in your logs. This channel behaviour comes from Nemotron 3.5 Lightning and the nemotron_v3 parser; the text donor does the same thing under an A/B on both endpoints.

Reasoning and the answer share the max_tokens budget. At max_tokens: 2048 and temperature 1.0, image requests with thinking on returned empty content 3 times in 8, each with finish_reason: length and a full reasoning channel — median reasoning was 1984 tokens against the 2048 cap. Either raise the budget or send enable_thinking: false on that path. See Best Practices.

We recommend thinking mode for tasks that involve reasoning and complex understanding. For video, audio, and omni use cases, try both enabling and disabling thinking for best results.

---

<a id="notes"></a>

Notes

1. frequency_penalty: 0.1 is required on every request, in every sampling mode. It is a request parameter, not a launch flag.
2. Reasoning default: reasoning is on by default and lands in message.reasoning, leaving content null. Send enable_thinking: false if your client reads content.
3. Audio input format: input_audio.data takes raw base64, not a data: URL.
4. Thinking preamble on images: with thinking left on, image requests may open with a "Here's a thinking process:" preamble. Set enable_thinking: false for direct image description.
5. Tool calling requires --enable-auto-tool-choice and --tool-call-parser at launch. Without them a request carrying tools returns HTTP 400 on every turn. Detect this from the engine's own 400 rather than from a hostname — the same engine can be relaunched with a parser.
6. max_tokens vs --max-model-len: max_tokens caps only the completion. It cannot exceed the server's --max-model-len, which is the ceiling for prompt + completion combined.
7. no_repeat_ngram_size does not exist in vLLM 0.27.1 or SGLang. Do not plan around it.

---

Training, Testing, and Evaluation Datasets:

Training Dataset:

No training was performed. This model is a merge of two published NVIDIA checkpoints; every weight is NVIDIA's, copied byte-for-byte from the source safetensors. For the training data behind those weights, see the model cards for Nemotron 3.5 Lightning and Nemotron 3 Nano Omni.

Data Modality: N/A · Data Collection Method: N/A · Labeling Method: N/A

Evaluation Dataset:

Benchmark Scores:

Text quality against Nemotron 3.5 Lightning, the text-tower donor. Paired design: both models saw identical items in identical order under identical sampling (temperature 0.0, frequency_penalty 0.1) in a single process, so the comparison is the per-item difference rather than two independent accuracy estimates.

Four paired comparisons across three tasks and three independently-built harnesses, 764 item-pairs, deltas straddling zero and none separable at p < 0.05. Item-level agreement — the two models returning the same verdict on the same problem — runs 94–98%.

On HumanEval the failure modes are near-identical: this model 18 AssertionError + 2 IndexError, the donor 17 + 1. Both fail by getting logic wrong rather than by emitting malformed or truncated code.

Multimodal capability, against ground truth. Every item was also sent to the text-only donor, which returns HTTP 400 "not a multimodal model" on all of them, so these results are not text priors:

Performance benchmarks, measured against stock Nemotron 3.5 Lightning NVFP4 under one harness driving both endpoints in the same session, DSpark n=3 on both, max_tokens 2048, enable_thinking: false. Served rate is sum(completion_tokens) / wall_clock_seconds:

The ratio changes sign with concurrency, in both regimes. Single-stream decode is memory-bandwidth-bound, and this checkpoint moves ~9.5% more weight bytes per decode step than the text-only donor; at concurrency 8 those reads amortise across the batch and this model comes out ahead — carrying vision and audio towers the donor does not have.

Measured on separate DGX Spark nodes, each engine in its resident configuration: gpu_memory_utilization 0.65 here, 0.45 for the donor; max_model_len 262,144 and 16,384. Neither approaches its KV limit at these sequence lengths.

Stock Nemotron 3.5 Lightning NVFP4 reaches 94.72 tok/s single-stream on this hardware, against its published 124.24 — that figure is not reproducible on a DGX Spark GB10 for either model.

Speculative decoding, DSpark n=3, per-position acceptance:

The drafter was trained against stock Lightning and runs here unmodified against a merged model. Acceptance sits within ~1.5 points of stock at every position and concurrency.

Observed in production, for contrast. The table above is a controlled benchmark. Over 20 h of continuous live agent traffic on a single DGX Spark GB10 — 91,458 drafts, 274,374 draft tokens — the same engine and drafter measured 82.44% acceptance and 3.473 accepted tokens/step, per-position 92.8 / 80.9 / 73.7.

The two figures are not in conflict; they are different workloads. Agent traffic — tool calls, structured output, code — is far more predictable than open-ended generation, so the drafter lands more of its proposals. Read the table as the controlled figure and this as the workload-dependent ceiling.

Two caveats, because a number without its conditions invites the same mistake in reverse:

• Temperature is uncontrolled here. Callers used whatever sampling they used across those 20 h. The table's rows state t=0.0 and t=1.0 explicitly; this row cannot, and some of the gap is likely sampling rather than workload.
• Same engine, same config. A spec-off baseline taken on this deployment measured 79.73 tok/s at c=1 against this card's 80.76 — 98.7% agreement, which is what licenses comparing the two rows at all. Absolute throughput gains do not port across harnesses even when the baseline does: the paired on/off delta on that harness was +28.7% at c=1 and +23.1% at c=8, against this card's +13.2% / +12.4% measured on Nik's. Different prompts, max_tokens, and cache state — the mechanism is the same, the percentage is not portable.

This model's acceptance figures are gated by the shim's tensor-level verification, which refuses to serve on mismatch and refuses to certify when it has nothing comparable to check. The stock control's drafter loads through vLLM's own DSpark path, which carries no equivalent gate; its evidence of correct loading is behavioural.

Quantization Benchmark Scores:

The NVFP4 recipe quantizes the MoE and dense GEMM weights to NVFP4 — FP4 E2M1 values with per-block FP8 E4M3 scales over groups of 16 elements and a per-tensor FP32 global scale — across 5,935 modules, with activations at 16-bit (W4A16). The Mamba mixer in_proj / out_proj are quantized to FP8 across 46 modules, and the KV cache is FP8. The SSM state path (conv1d, gate), the attention projections on the six KV-carrying layers, the embeddings and the mtp tensors are kept at higher precision, as are the vision and audio encoders and their MLP projectors — 72 excluded modules in total. This yields 5.69 effective bits per weight (24.4 GB); over the NVFP4 portion alone the rate is exactly 4.50 bits per weight, being 4 bits of value plus one FP8 scale byte per 16 elements.

The difference in bits-per-weight is the modality towers: 3.27B of this checkpoint's parameters sit in BF16 tensors, of which 1.44B are the vision and audio encoders and their projectors.

Benchmarks evaluated: GSM8K, MMLU, HumanEval; ASR against a verbatim transcript; image, video and audio capability probes against ground truth.

Data Collection Method by dataset: Hybrid: Human, Automated — public academic datasets with automated scoring.

Labeling Method by dataset: Human.

Properties: GSM8K sampled from openai/gsm8k main/test; MMLU sampled from cais/mmlu all/test across all subjects; HumanEval evaluated on all 164 problems. All comparisons paired within a single process against a live endpoint of each model. Each harness was made to fail on purpose before its results were used — the throughput harness against a dead port and a bogus model name, the scorer against correct, malformed, wrong and empty responses, and the HumanEval grader against 20 canonical solutions (requiring 20/20 pass) and the same 20 corrupted (requiring 20/20 fail). Zero completions reached the token cap on HumanEval (1536) or GSM8K (1024); one did on MMLU (512).

Inference:

Acceleration Engine: vLLM 0.27.1, with DSpark speculative decoding at n=3

Test Hardware:

• NVIDIA DGX Spark (GB10)

<a id="best-practices"></a>

Best Practices

These are the settings it was measured at and the ones we would start from.

Sampling Parameters

• Send frequency_penalty=0.1 on every request, in every mode.
• Thinking mode for document analysis and multimodal reasoning: temperature=0.6, top_p=0.95, frequency_penalty=0.1
• Instruct mode (non-thinking) for general tasks: temperature=0.2, top_k=1, frequency_penalty=0.1
• For ASR: non-thinking mode, temperature=0.2, top_k=1, frequency_penalty=0.1

Model output length

Reasoning and the final answer draw on the same max_tokens budget, so a budget sized for the answer alone will truncate before the answer begins — the request returns finish_reason: length with a populated reasoning channel and content empty.

Measured empty-content rates at max_tokens=2048 with thinking on: 0% on short conversational prompts (p95 reasoning 923 tokens), 12.5% on ordinary factual and short-explanation prompts, and 37.5% on image prompts, whose median reasoning was 1984 tokens against the 2048 cap.

So: 2048 is sufficient with thinking off, and sufficient with thinking on for short conversational turns. For anything harder, use 4096 or more, or send enable_thinking: false. For real-time voice, enable_thinking: false is also the latency choice — on a 6.24 s clip it produced a byte-identical transcription in 0.52 s against 11.11 s with thinking on.

Ethical Considerations:

This is a derivative model. Developers should work with their own teams to ensure it meets the requirements of their industry and use case, and should evaluate its safety properties directly rather than relying on evaluations published for the source models.

Please make sure you have proper rights and permissions for all input image, audio and video content.

For ethical considerations relating to the source models, see NVIDIA's Model Card++ subcards for Nemotron 3 Nano Omni and Nemotron 3.5 Lightning.

<a id="license"></a>

License

This model is a derivative work built from NVIDIA checkpoints published under two different licenses. Both apply, to different parts of the model, and both must be complied with by anyone redistributing it.

Full license texts are included in this repository as LICENSE.Omni-NVIDIA-Open-Model-Agreement.pdf and LICENSE.Lightning-OpenMDW-1.1. Attribution notices are in NOTICE. Tensor-level provenance is in PROVENANCE.md.

Third-party code loaded at runtime

This model's config.json references nvidia/C-RADIOv2-H via auto_map for the vision-tower implementation. Loading this model with trust_remote_code=True will fetch that code directly from NVIDIA. It is distributed by NVIDIA under the NVIDIA Open Model License Agreement (June 14, 2024), a separate and older agreement from the one governing this model. It is not redistributed here; your use of it is governed directly by NVIDIA's terms.

Attribution

This model would not exist without NVIDIA's decision to publish the Nemotron family under permissive terms. All pretrained weights are NVIDIA's. No weight in this model was trained, fine-tuned, recomputed, or requantized here — the merge is a header-level graft of NVIDIA's published tensors, copied byte-for-byte.

Redistributed code retains the copyright notices of its original authors, including NVIDIA Corporation, AI21 Labs Ltd., the HuggingFace Inc. team, and the Qwen team at Alibaba Group. See NOTICE.

Obligations passed downstream

If you redistribute this model or a derivative of it, you must:

1. Include a copy of both license texts with your distribution.
2. Retain all copyright, patent, trademark, and attribution notices in the source form of any derivative you distribute (NVIDIA Open Model Agreement §3(b)).
3. Retain all copyright notices and notices of origin (OpenMDW-1.1).
4. Comply with all applicable export, import, trade and economic sanctions laws and regulations, including the U.S. Export Administration Regulations and Office of Foreign Assets Control regulations (NVIDIA Open Model Agreement §10).

Note that both licenses terminate your grant if you initiate patent or copyright litigation concerning these models.

No warranty. Both upstream licenses provide these materials "AS IS", without warranty of any kind. This derivative is provided on the same basis.

<a id="speculative-drafter"></a>

Speculative Drafter

Every throughput figure above was measured with the DSpark drafter attached. It ships separately, with the plugin shim that lets it attach:

brazley/Brazley-Nemotron-3.5-Lightning-Omni-Drafter

Citation:

@misc{brazley2026lightningomni,
      title={Brazley Nemotron 3.5 Lightning Omni: a multimodal merge of
             NVIDIA Nemotron 3.5 Lightning and NVIDIA Nemotron 3 Nano Omni},
      author={Brazley, Nik},
      year={2026},
      howpublished={\url{https://huggingface.co/brazley/Brazley-Nemotron-3.5-Lightning-Omni}},
      note={Not an NVIDIA release}
}