distilbart-mnli-12-1-onnx

distilbart-mnli-12-1, ONNX (INT8 per-channel)

An ONNX export of valhalla/distilbart-mnli-12-1, dynamically quantized to INT8 per-channel, for zero-shot classification in transformers.js / onnxruntime-web.

Produced for web2music, where it is an opt-in alternative to the shipped default (Xenova/nli-deberta-v3-xsmall).

Why this exists

The commonly used mirror Xenova/distilbart-mnli-12-1 now returns 401 Unauthorized to anonymous downloads (as does its -12-3 sibling) — gated or removed upstream. This is a fresh export from the original valhalla checkpoint, which is still public.

Files

Quantization tradeoff — measured, and not in this model's favour

INT8 dynamic quantization measurably hurts this checkpoint. A 7-sentence full 13-way ranking check against eager PyTorch fp32 ground truth:

Per-channel is usable but degraded. Do not assume parity with the fp32 model. This is a small-sample spot check, not a rigorous benchmark — treat it as a warning, not as a calibrated accuracy figure.

Converting it required working around three real bugs

All three are unrelated to this specific checkpoint and worth knowing about if you are exporting anything else on Python 3.14 / torch 2.12 / Windows:

1. PyPI optimum / optimum-onnx wheels ship an empty exporters/onnx/ tree, regardless of version pairing. Install optimum from GitHub source at a matching tag instead of from PyPI.
2. Python 3.14 made functools.partial a descriptor, which breaks optimum's NORMALIZED_CONFIG_CLASS = SomeConfig.with_args(...) pattern — accessing it via an instance now silently binds self as an extra positional argument.
3. torch 2.12's torch.onnx.export defaults to the symbolic-shape-proving torch.export tracer, which correctly refuses to trace BartForSequenceClassification as shipped. Its EOS-token pooling (hidden_states[input_ids.eq(eos_token_id), :]) is genuinely value-dependent. Forcing the legacy tracer instead "succeeds" by silently baking in wrong trace-time assumptions — verified: confidently backwards entailment/contradiction logits on real inputs, present even in fp32, before any quantization. The real fix replaces the pooling with a static equivalent (attention_mask.sum(dim=1) - 1, numerically verified identical to the original, max diff 0.0), which lets the proving tracer succeed instead of the silently-wrong one.

The conversion script and both patches are in the web2music repo under mood-classification/experiments/ (convert_bart_mnli_to_onnx.py, _optimum_py314_shim.py, _bart_seqcls_onnx_patch.py).

Usage

import { pipeline } from '@huggingface/transformers';

const classifier = await pipeline(
  'zero-shot-classification',
  'AyushK0808/distilbart-mnli-12-1-onnx',
);
const out = await classifier('This page is about hiking in the Alps.', [
  'travel', 'news', 'shopping',
]);

License

valhalla/distilbart-mnli-12-1 declares no license on the Hub. Its ancestor facebook/bart-large-mnli is MIT. This export adds no license of its own and claims no rights beyond the upstream weights — check with the original authors before relying on it in a context where licensing matters.