PersonaPlex-7B-ONNX

PersonaPlex 7B ONNX

ONNX-quantized bundles of NVIDIA's PersonaPlex 7B — a full-duplex speech-to-speech model on Kyutai's Moshi architecture. Listens and speaks simultaneously at 12.5 Hz, conditioned on a voice preset and a text system prompt.

This repository ships four production-ready bundle variants spanning the disk × host RAM × VRAM × RTF × quality trade-off space. Pick one based on your target hardware and quality bar.

Variants at a glance (measured on RTX 5090, 50 frames, VARF2 voice, "helpful" prompt)

RTF (real-time factor) is per-frame latency / frame interval at 12.5 Hz — 1.0× = exactly realtime, < 1.0× = faster than realtime. The int8-nb-dep_gint8 and int4-nb-dep_gint8 variants run at ~1.12× (near-realtime streaming).

Which variant to pick

• Most use cases → int8-nb-dep_gint8: best balance of RTF, host RAM, and quality. If your GPU has ≥16 GB VRAM, this is what you want.
• Limited VRAM (≤8 GB GPU) → mixed: only 6.6 GB VRAM. Costs host RAM and RTF, but quality is still excellent (cos 0.990) with the best topical responses on our benchmark.
• Disk-constrained → int4-nb-dep_gint8: 7.6 GB on disk and only 9.6 GB VRAM. Accept some quality drift (cos 0.877 — coherent English but less precise).
• Maximum quality regardless of cost → fp16: cos 0.9999, indistinguishable from the FP32 reference.

Architecture

[User audio 24 kHz PCM]
        ↓
[Mimi encoder: SEANet + 8L transformer + RVQ] → 16 codebooks @ 12.5 Hz
        ↓
[Temporal transformer: 32L, dim=4096, 7B params, RoPE, RMSNorm, SwiGLU]
        ↓
[Depformer: 6L, dim=1024, MultiLinear × 16 codebook steps] → 16 agent audio tokens
        ↓
[Mimi decoder] → 24 kHz agent audio PCM

Each variant ships four ONNX graphs:

Plus per-variant auxiliary files:

How to use

Via the C++ runtime (speech-core)

# Download the recommended variant
PERSONAPLEX_VARIANT=int8-nb-dep_gint8 scripts/download_personaplex_onnx.sh

# Run end-to-end
build/Release/run_personaplex scripts/personaplex-int8-nb-dep_gint8 50 \
    tests/data/test_audio.wav VARF2

Via ONNX Runtime in Python

import onnxruntime as ort

# Inputs:  text_token [1,1] int64
#          audio_tokens [1,16] int64
#          past_k_all [32, 1, 32, T_past, 128] float (FP32 or FP16 depending on bundle)
#          past_v_all (same shape)
# Outputs: hidden [1, 1, 4096]
#          new_k_all [32, 1, 32, T_full, 128]
#          new_v_all (same shape)
sess = ort.InferenceSession("temporal_step.onnx",
                             providers=["CUDAExecutionProvider"])

For full-duplex generation, also call depformer_step 16 times per frame (one inner step per audio codebook) — see the speech-core wrapper source for the complete loop.

How these bundles were produced

All four bundles export from the FP32 PyTorch reference via stages in convert_onnx.py:

Mimi codec is FP32 in all four variants (small enough not to matter).

Related

• soniqo/speech-core — C++ inference runtime with the OnnxPersonaPlex wrapper, CUDA EP routing, multi-turn KV cache, and 12 memory-tuning env knobs (SPEECH_CORE_USE_ENV_ALLOCATORS, etc.)
• soniqo/speech-models — model export pipeline including convert_onnx.py, quantize_depformer_gather.py, bench_pytorch_cuda.py, compare_bundle_quality.py
• soniqo.audio — the project site
• nvidia/personaplex-7b-v1 — upstream PyTorch reference
• kyutai-labs/moshi — base architecture

License

NVIDIA Open Model License (same as upstream). See the LICENSE link.