nemotron-speech-streaming-en-0.6b-aimodel

Nemotron Speech Streaming EN 0.6B — Core AI

Community conversion by spybyscript of NVIDIA's original English checkpoint into Apple's Core AI .aimodel format for OS 27. Exported directly from the original PyTorch weights, with no training or fine-tuning. This is not an official NVIDIA or Apple release.

Each variant contains 31 component assets for a stateful 560 ms English RNN-T pipeline, plus tokenizer, tensor contract, checksums and validation. A host decoder carries the caches and turns token IDs into text. The included Swift streaming runner is the tested reference implementation; see INTEGRATION.md.

Related formats: spybyscript's Android LiteRT conversion. The Core AI export uses the original checkpoint, not the LiteRT files. This is the English model, not Nemotron 3.5.

Source and license

• Original revision: ebe59e5a817142986528bbbee5dba8db7b38ed50.
• Source safetensors SHA256: bddd8a7300826efd19cf7e01f1c7db8402bed6786fc4c7739632894f69c71473.
• Model materials retain the NVIDIA Open Model License; attribution is in NOTICE.md.
• The upstream card is preserved in SOURCE_MODEL_CARD.md. Its benchmarks are upstream results, not measurements of this conversion.
• Export: PyTorch 2.11.0, Transformers 5.17.0, coreai-torch 0.4.2, coreai-core 1.0.0b2, coreai-opt 0.2.1. See conversion/README.md.

Variants and measured results

The FP32 native reference was independently cross-checked against the original model's generate() with the same 560 ms chunk profile. Conversion evaluation used 20 selected LibriSpeech test-clean recordings, 10 speakers, 171.555 seconds, 488 reference words. This is a small conversion test, not the full LibriSpeech benchmark or broad deployment qualification.

¹ Bytes recorded by each manifest, excluding the manifest itself. Decimal GB. ² Peak resident memory for the entire Swift benchmark process, not just model weights or a prediction of iPhone memory. ³ Processing seconds / audio seconds, excluding the first recording; lower is faster. These warm timings do not include loading, initial specialization or audio capture. Median per-recording chunk p50 was 42.7 ms (FP16) and 47.3 ms (W8), per 560 ms model chunk. The full report includes first-recording costs, per-recording timings and numerical drift.

• FP16 is the initial integration baseline. Network weights, activations and caches are half precision. The waveform frontend stays FP32; lowering its DFT/mel arithmetic to FP16 increased frontend error in initial tests.
• W8 is a size-focused experimental alternative. Apple's per-channel symmetric INT8 weight quantization is applied to the 24 encoder blocks; activations/caches and remaining network components stay FP16, frontend FP32. It reduces bundle size by about 48% and process memory here, but was slightly slower on this Mac. This is not a claim of integer-only or Neural Engine execution.
• W8 failed exact-token parity on two recordings. Its lower WER on this tiny sample does not establish an accuracy improvement. The failed gate and changed hypotheses remain in w8/validation.json. Explicit opt-in is required to run it.

Full results: FP16, W8. Evaluation transcripts and audio hashes are included; the recordings themselves are not distributed. See corpus attribution.

Runtime and sustained validation

Tested on Apple M5 Max, 128 GiB RAM, macOS 27.0 (26A5425a), Xcode 27.0 (27A266a), Swift 6.4. Core AI default specialization was used. The benchmark performs host copies between graphs and is a correctness-oriented reference, not a fully fused implementation. Actual GPU/CPU/Neural Engine placement was not profiled.

The standalone Swift implementation also completed 80 utterances per variant in one process (686.22 seconds of audio): the same 20 recordings repeated four times with varying callback sizes, complete cache resets and an empty-stream check. All outputs matched that variant's captured results. Peak RSS was 1.44 GB FP16 / 1.02 GB W8. Retained PCM after callbacks stayed below one 9,313-sample model window. This is a repeated short-utterance stability test, not a new 80-recording accuracy set or a single uninterrupted 11-minute utterance. See FP16 stability and W8 stability.

The beta Python Core AI wrapper repeatedly aborted after eight recordings with an IOSurface allocation failure. Autorelease pools, direct buffer copies and forced garbage collection did not resolve it. Swift with explicitly reusable input/output buffers passed the sustained test. The Python wrapper's underlying cause remains unresolved; use the included Swift runner for sustained testing.

Both final variants were compiler-checked for iOS 27, architecture h19p. Compiler warnings prevent any claim of confirmed Neural Engine placement. No iPhone/iPad execution, energy, thermal or microphone latency measurements are claimed. These universal .aimodel source assets still specialize on the destination device; compiler outputs are not distributed. See compilation evidence.

Download and transcribe on a Mac

Requirements: Apple silicon, macOS 27, Xcode 27 and Apple's Metal Toolchain. Model execution uses the system Core AI framework; no Python, Transformers, NeMo, LiteRT or ONNX runtime is needed by the Swift executable.

Download one variant with Hugging Face's CLI (pin an immutable repository revision for deployment):

hf download spybyscript/nemotron-speech-streaming-en-0.6b-aimodel \
  --include 'fp16/*' '*.swift' '*.md' 'LICENSE.pdf' 'publication.json' 'SHA256SUMS' \
  --local-dir nemotron-en-aimodel
cd nemotron-en-aimodel
xcodebuild -downloadComponent MetalToolchain
xcrun swiftc -parse-as-library -O CoreAISpeechRunner.swift SwiftTranscribe.swift -o transcribe
./transcribe fp16 recording-mono-16khz.wav

The executable verifies bundle hashes before loading, prints JSON containing text, token IDs, timing and memory, and streams the file through deliberately unaligned 997-sample callbacks. Resample/downmix to mono 16 kHz before invocation. Keep .aimodel directories intact.

For W8, download w8/* as well, then explicitly opt in:

./transcribe w8 recording-mono-16khz.wav --allow-experimental

For an app, keep graphs and tensor storage resident, serialize stream operations, and reset all caches between utterances. SpeechRunner.feed(_:final:) accepts incremental Float32 PCM; reset() starts a new utterance. The reference currently copies activations through host arrays and uses the main actor for serialization. Move app inference to a dedicated execution context and validate end-to-end Swift transcripts before adding a UI. VAD, diarization, resampling and speaker segmentation are not included.

Limitations and next checks

This conversion preserves a streaming ASR model's existing recognition weaknesses; format conversion does not improve its speech understanding. Names, emotional speech, overlap, music, noise and omitted words need representative evaluation. Token parity is scoped to the tested recordings. W8 especially needs a larger, diverse accuracy test before becoming a default. Measure phone memory and latency with concurrent emotion analysis, cold versus warm operation, thermal load and energy before choosing a shipping configuration.

Apple references: Core AI guide, PyTorch converter, Core AI models and Swift examples, optimization library.