reason-mxbai-colbert-v0-32m-onnx

Reason-mxbai-colbert-v0-32m for Transformers.js

This repository is a browser-compatible ONNX export of DataScience-UIBK/Reason-mxbai-colbert-v0-32m, a roughly 32M-parameter, English late-interaction retrieval model trained for reasoning-intensive search. Use the upstream repository for PyTorch and PyLate; use this repository when you need to run the model with @huggingface/transformers and ONNX Runtime, including inside a web browser.

This is a feature-extraction model, not a hosted inference endpoint. It does not directly return passages or answers. It produces one normalized, 128-dimensional vector per input token. A retrieval application must store the document vectors and compare them with query vectors using ColBERT MaxSim. MaxSim lets each query-token vector select its strongest document-token match, then combines those matches into one relevance score for the passage.

At a glance

What was exported

The upstream model is a fine-tuned version of mixedbread-ai/mxbai-edge-colbert-v0-32m. Its training uses reasoning-oriented data and hard negatives, and its projection head emits 128 values per token. This repository converts that model to ONNX; it does not retrain it or change its retrieval objective.

Files in this repository:

• model.onnx is the FP32 reference export and the recommended browser model.
• model_int8.onnx is an experimental reduced-range dynamic INT8 export. It kept the same winner in the small test below, but quantization moved individual vector values more substantially and can reorder close results.
• validation.json contains the exact query, corpus, reference judgment, scores, rankings, and conversion-error measurements used below.
• tokenizer and configuration files come from the upstream checkpoint.

Browser usage

import { AutoModel, AutoTokenizer } from "@huggingface/transformers";

const modelId = "devame/reason-mxbai-colbert-v0-32m-onnx";

const [tokenizer, model] = await Promise.all([
  AutoTokenizer.from_pretrained(modelId),
  AutoModel.from_pretrained(modelId, {
    dtype: "fp32",
    subfolder: "",
  }),
]);

async function encode(text, kind) {
  // Lowercase the content first. Keep the special prefix uppercase.
  const prefix = kind === "query" ? "[Q] " : "[D] ";
  const inputs = await tokenizer(`${prefix}${text.toLocaleLowerCase()}`, {
    padding: true,
    truncation: true,
  });
  const result = await model(inputs);
  return result.output; // [batch, sequence length, 128]
}

const queryVectors = await encode(
  "Who approved the Atlas deployment?",
  "query",
);

The case-sensitive [Q] and [D] prefixes are part of the model's expected input format. Lowercasing the entire final string would turn them into [q] and [d] and change the input the model sees. Padding and special-token vectors must also be masked appropriately when implementing MaxSim.

Validation

The validation below answers two separate questions:

1. Illustrative retrieval behavior: how does this reasoning-trained model order one deliberately difficult four-passage corpus compared with a named lightweight late-interaction model?
2. Export fidelity: does this ONNX export reproduce its own upstream PyTorch source model on the same inputs?

This single example is a smoke test, not evidence of general retrieval quality.

Models in the comparison

• Comparison retriever: mixedbread-ai/mxbai-edge-colbert-v0-17m, a 17M-parameter late-interaction model with 48-dimensional token vectors. It supplies behavioral context because it was already used in the browser experiment that motivated this export. It is not the parent of the 32M model.
• PyTorch reference: DataScience-UIBK/Reason-mxbai-colbert-v0-32m, the source checkpoint from which this repository was exported.
• Export under test: this repository's FP32 model.onnx, loaded with @huggingface/transformers.

These models differ in size, backbone, projection width, and training. The comparison therefore illustrates their behavior on this case; it is not a controlled experiment isolating reasoning training.

Query and complete retrieval corpus

Query: Who approved the Atlas deployment?

The following block is the entire input corpus. Its four passages appear in their original order and are separated only by blank lines. The models did not receive passage numbers, labels, or commentary.

The Atlas deployment checklist named Elena Ortiz as release manager. Elena
reviewed the checksum report but did not approve the deployment after it
passed; she had already delegated that decision to the on-call engineer. The
audit log records her review and the completed checksum verification.

Ravi Shah was the on-call engineer for the Atlas release. After the checksum
passed and the rollback snapshot finished, he approved the deployment at
21:14 UTC. Elena Ortiz reviewed his decision the following morning.

The Orion deployment was approved by Elena Ortiz after its checksum passed.
That release used the same checklist later adopted by the Atlas team, although
it had a different on-call rotation and rollback procedure.

Checksum verification compares a release artifact with its expected digest.
A successful match permits the responsible release engineer to continue, but
it does not identify which person has approval authority for a particular
deployment.

Reference judgment

Expected answer: Ravi Shah, supported by the second passage.

The second passage identifies Ravi Shah as the Atlas on-call engineer and then states that he approved the deployment. The first passage explicitly says Elena did not approve Atlas. The third says Elena approved Orion, not Atlas. The fourth explains checksum verification without naming an approver.

Retrieval result

Passage numbers in this table refer only to paragraph order in the corpus above. Each model used its expected query/document preprocessing, and every column used the same ColBERT MaxSim calculation. The PyTorch reference and this ONNX export received identical lowercased [Q]/[D] inputs.

On this corpus, mixedbread-ai/mxbai-edge-colbert-v0-17m ranks the third passage first, even though it concerns Orion. The upstream reasoning-trained 32M model ranks the second passage—the one supporting Ravi Shah—first. Scores from different models are not calibrated against each other, so compare rank order within a column rather than comparing, for example, 0.976861 from one model with 0.953260 from another.

ONNX conversion fidelity

The PyTorch source model and FP32 ONNX export produce the same six-decimal MaxSim scores and the same passage order: 2, 3, 1, 4. Across the checked batch, the largest difference between any individual PyTorch and ONNX embedding value was 1.52e-6. A fresh load through @huggingface/transformers reproduced the same scores and ranking.

This establishes conversion parity for these tested inputs. It does not prove that the reasoning-trained model is generally better than the 17M comparison model. That claim would require a held-out corpus and aggregate measurements such as Recall@k, MRR, or nDCG. See the validation.json file for the machine-readable fixture and results, and the upstream model card for benchmark results such as BRIGHT nDCG@10.

Checksums

model.onnx       1902B0E650E0CAFD52F263C192950A1205DF2C316E3C1C2596651B625335BD7B
model_int8.onnx  9970B66D0199463B6AA394722271E072C1ECE434383B0F54BF554F6724EB4B4D

Provenance and license

The export was created for an devame/blog browser retrieval experiment. Model authorship, training details, datasets, evaluations, and citations belong to the DataScience-UIBK/Reason-mxbai-colbert-v0-32m source repository.

The upstream repository currently contains conflicting license signals: its metadata and final license heading say Apache-2.0, while its Model Details section says CC-BY-NC-4.0 is inherited from the training data. This derivative repository uses the more restrictive CC-BY-NC-4.0 designation unless and until the upstream authors clarify that inconsistency.