fireredpunc-onnx

fireredpunc-onnx

Weight-only 8-bit ONNX export of FireRedPunc, FireRedTeam's Chinese/English punctuation restoration model, mirrored for use by Sokuji's local sentence-segmentation stage.

Upstream

• Weights: FireRedTeam/FireRedPunc — Apache-2.0.
• Code: FireRedTeam/FireRedASR2S @ 4e7d9aaf, fireredasr2s/fireredpunc — Apache-2.0 (LICENSE in this repo is the verbatim upstream text).
• Base model: chinese-lert-base (BERT-base, 12 layers, vocab 21,128) with a token classifier over out_dict = {<space>, ，, 。, ？, ！}.
• The int8 ONNX export published alongside the weights (42ailab/FireRedPunc-ONNX) uses dynamic per-tensor int8 quantization, which loses about half of the sentence-final marks (see "Why not the published int8 file" below). This repo is not that file — it is a fresh weight-only 8-bit export built from upstream's fp32 weights.

Files

tokenizer.json is byte-identical to upstream's chinese-lert-base/tokenizer.json; its vocab is identical to the chinese-bert-wwm-ext_vocab.txt upstream uses for token ids. out_dict names the five output classes in punc.q8w.onnx's label order.

Conversion

Built with onnxruntime.quantization.matmul_nbits_quantizer.MatMulNBitsQuantizer from a fp32 ONNX export of the upstream PyTorch weights (fireredasr2s.punc.ModelIO), 8 bits, block size 32, symmetric — the same recipe FireRedPunc's own weight-only build in this benchmark uses, and the one weight-only 8-bit build that reproduced upstream's decisions exactly (see below). Every activation (input_ids, attention_mask, logits) stays float32; only the MatMul weights are quantized.

Exact command, run against Sokuji's punctuation benchmark harness (benchmark/punctuation-restoration/):

python parity/fireredpunc-quant.py

This writes punc.{int8pc-nocls,q8w,q4w}.onnx into the working directory; punc.q8w.onnx here is that script's q8w output, copied unmodified.

Measured numbers

From docs/superpowers/notes/2026-09-14-asr-punctuation-benchmark.md (Sokuji repo, 2026-09-14):

• Parity vs. upstream PyTorch fp32, 153 zh/en rows / 8,798 tokens (parity/fireredpunc.py reference, parity/fireredpunc.mjs ported JS on onnxruntime-web 1.26 WASM): 153/153 rows exact, 100% of token classes, all 353 sentence-final marks placed correctly. The published dynamic-int8 file (42ailab) reaches only 43/153 rows and 96.8% of token classes against the same reference, because dynamic per-tensor int8 turns roughly half of sentence-final marks into commas.
• Quality (zh, stripped input): breakpoint F1 91.5 offline (GPT-Live raw transcript 90.8); sentence-end F1 55.4 (precision 93 / recall 39 — the model prefers ， to 。); streaming breakpoints P/R 93.0 / 90.2, streaming sentence ends P/R 84.4 / 53.5 (8-character right-context commit).
• Renderer cost (Electron 40.8.5, DGX Spark GB10 aarch64, WebGPU): 163 MB on disk, zh 480 characters in 27 ms, en 960 characters in 19 ms; WebGPU output identical to 4-thread WASM output on all 57 corpus inputs. On WASM alone: zh 394 ms / en 246 ms at the same lengths.
• English text run through this model comes back lowercased (RuleBaedTxtFix lowercases everything and only re-capitalizes sentence starts and "I"); Sokuji restores casing from the ASR input rather than trusting this model's output casing.

License

Apache-2.0, carried over from FireRedTeam/FireRedPunc and FireRedTeam/FireRedASR2S. See LICENSE in this repository (verbatim copy of the upstream FireRedASR2S LICENSE file).