Nanbeige4.2-3B-ONNX

Nanbeige4.2-3B-ONNX

ONNX export of Nanbeige/Nanbeige4.2-3B for cross-browser inference via ONNX Runtime Web.

This is the companion to nicolasembleton/Nanbeige4.2-3B-GGUF (native/server-side via llama.cpp, Ollama, LM Studio).

Architecture note

Nanbeige uses a loop transformer (num_loops=2 — two passes per physical layer). Stock ONNX Runtime Web doesn't have a MatMulNBits loop unroller for this. We solve it by unrolling the loop at the Python level: 44 sequential layer calls share 22 weight matrices. The exported graph is a standard ONNX opset-18 graph that runs in stock ONNX Runtime Web and transformers.js — no custom kernels needed.

Validation: bit-exact match against the stock PyTorch model.

Files

• model.onnx — 1.8 MB graph
• model.onnx_data — 4.0 GB consolidated BF16 weights
• config.json, tokenizer*, vocab.json, etc.

Browser usage (cross-browser, including Apple Safari)

import * as ort from "onnxruntime-web";

const session = await ort.InferenceSession.create(
  "https://huggingface.co/nicolasembleton/Nanbeige4.2-3B-ONNX/resolve/main/model.onnx",
  { executionProviders: ["webgpu", "wasm"] },  // Safari 17 macOS falls back to WASM
);

const tokens = [166100, 1234, 5678];  // your token ids
const feeds = {
  input_ids: new ort.Tensor("int64", BigInt64Array.from(tokens.map(BigInt)), [1, tokens.length]),
  attention_mask: new ort.Tensor("int64", BigInt64Array.from(tokens.map(() => 1n)), [1, tokens.length]),
  position_ids: new ort.Tensor("int64", BigInt64Array.from(tokens.map((_, i) => BigInt(i))), [1, tokens.length]),
};
const { logits } = await session.run(feeds);

Alternative: transformers.js

import { pipeline } from "@huggingface/transformers";

const generator = await pipeline(
  "text-generation",
  "nicolasembleton/Nanbeige4.2-3B-ONNX",
  { device: "webgpu" },  // or "wasm"
);
const output = await generator("Hello, how are you?", { max_new_tokens: 256 });

Note: This model is prefill-only (forward pass, no KV cache baked in). For autoregressive generation you'll need to feed inputs back through and argmax over logits. KV-cache export is a future enhancement.

Safari note: Safari 17+ on macOS Sonoma supports partial WebGPU. iOS Safari has no WebGPU — use the WASM execution provider (slower but works). Node.js also works via WASM.

License

Apache 2.0 (inherited from Nanbeige/Nanbeige4.2-3B).

Citation

@misc{nanbeige42-3b-onnx,
  title = {{Nanbeige4.2-3B-ONNX}},
  author = {{nicolasembleton}},
  year = {{2026}},
  howpublished = {{Hugging Face}},
  note = {{Cross-browser ONNX export with Python-level num_loops=2 unroll. BF16, 4 GB.}},
}}