personaplex-7b-turbo2bit

PersonaPlex 7B — TurboQuant 2-bit (NF2 + Walsh-Hadamard)

Ultra-compressed version of nvidia/personaplex-7b-v1 for deployment on edge devices like NVIDIA Jetson AGX Orin.

Quantized using a TurboQuant-inspired method combining Walsh-Hadamard Transform rotation with Normal Float 2-bit (NF2) optimal scalar quantization.

Compression Results

Quantization Method

TurboQuant v2 — adapted from Google Research TurboQuant (arXiv:2504.19874):

1. Per-group normalization (group_size=128): RMS scaling to unit variance
2. Walsh-Hadamard Transform: Random rotation to spread weight outliers uniformly across dimensions, making the distribution more Gaussian
3. NF2 quantization: Lloyd-Max optimal 4-level scalar quantizer for Gaussian distributions
   • Centroids: {-1.5104, -0.4528, 0.4528, 1.5104}
   • Decision boundaries: {-0.9816, 0.0, 0.9816}
4. Packed storage: 4 codes per byte (2 bits each) + fp16 per-group scales

What stays in fp16 (77 tensors): Layer norms, biases, embeddings, positional encodings, RoPE — these are sensitive to quantization and small enough to keep full precision.

Per-Tensor Quality

Target Hardware

PersonaPlex requires 12.5 frames/second (80ms/frame budget). At 2.07 GB, weight loading is well under budget on any modern GPU.

Quick Start — Native 2-Bit Inference

Keeps packed 2-bit weights on GPU, dequantizes per-matmul. ~10 GB peak VRAM vs 19 GB for bf16.

import torch
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download

# Download this repo's files
weight_path = hf_hub_download("cudabenchmarktest/personaplex-7b-turbo2bit", "model-turbo2bit.safetensors", token=False)
linear2bit_path = hf_hub_download("cudabenchmarktest/personaplex-7b-turbo2bit", "linear2bit.py", token=False)

# Import the Linear2bit module
import importlib.util
spec = importlib.util.spec_from_file_location("linear2bit", linear2bit_path)
linear2bit_mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(linear2bit_mod)

# Load model (requires moshi-personaplex package)
from moshi.models.loaders import _lm_kwargs, LMModel
lm_kwargs = dict(_lm_kwargs); lm_kwargs["dep_q"] = 16
model = LMModel(device="meta", dtype=torch.bfloat16, **lm_kwargs)
state_2bit = load_file(weight_path, device="cpu")
model = linear2bit_mod.replace_linears_with_2bit(model, state_2bit, device="cuda", dtype=torch.bfloat16)
model.eval()

Dequantize to BF16 (if you prefer standard weights)

python dequantize.py --input model-turbo2bit.safetensors --output model-bf16.safetensors
# Output: 15.59 GB bf16 file

Server with personaplex-setup

# Native 2-bit (~10GB peak, no HF token needed)
NO_CUDA_GRAPH=1 python -m moshi.server --moshi-weight model-turbo2bit.safetensors --native-2bit --device cuda

# + CPU Mimi codec (saves ~840MB)
NO_CUDA_GRAPH=1 python -m moshi.server --moshi-weight model-turbo2bit.safetensors --native-2bit --cpu-mimi --device cuda

Voice Cloning

python clone-voice.py --input my_voice.wav --name MyVoice --device cuda

File Structure

Quantization Hardware

Quantized on 3x NVIDIA A100 80GB PCIe in 50 seconds using CUDA-accelerated Walsh-Hadamard Transform.

Related Models

• nvidia/personaplex-7b-v1 — Original bf16 weights (15.59 GB)
• cudabenchmarktest/personaplex-7b-nf4 — INT4 NF4 quantized (4.14 GB, 99.4% cosine sim)

License

Same as base model: NVIDIA Open Model License

Credits

Quantized by open-agents-ai using TurboQuant-inspired NF2+WHT method. Base model by NVIDIA Research (PersonaPlex paper).