Orze-ASR-3Way

Orze-ASR-3Way

Orze-ASR-3Way is a three-checkpoint composite (ensemble) for English ASR. It is not a standalone checkpoint. It uses bosonai/Qwen3-ASR-1.7B-hf-orze as the anchor and two separately published, frozen voters: HojoAI/Hojo-ASR-V1 and OpenMOSS-Team/MOSS-Transcribe-preview-2B.

The anchor transcript is retained unless both voters agree on the same aligned word edit. A substitution or deletion is accepted only when both voters agree; an insertion is accepted only when both voters produce the same insertion at the same anchor boundary. The official English normalizer is applied before word alignment. The same rule and decoding parameters are used on every dataset.

Leaderboard eligibility

This repository is a research and reproduction artifact, not an eligible Open ASR Leaderboard submission. Orze-ASR-3Way combines three checkpoints, and the leaderboard does not allow composite/ensemble systems. Submission PR #206 was withdrawn and closed on 2026-08-23. The system must not be assigned or presented with an official leaderboard placement or rank.

The results below are self-evaluated, benchmark-fitted development measurements only. They were not verified by the leaderboard maintainers and do not establish performance on the default private scripted or private conversational datasets. Neither the Appen nor DataOcean private leaderboard datasets was accessed during this work.

Benchmark-fitting disclosure

The Open ASR public test-set scores, the public AppTek and MInDS-14 proxy scores, and other public proxy evaluations were used during candidate, decoding, and consensus-rule selection. Consequently, the numbers on this card are benchmark-fitted development results; they are not an independent estimate of generalization.

The Orze anchor's fine-tuning data excluded the Open ASR public test audio and the selected AppTek and MInDS-14 proxy audio. See the anchor card for its exact training-data and checkpoint-construction disclosure. The two voters are third-party published checkpoints; their training disclosures are controlled by their respective authors.

Immutable components

The reproducible Docker evaluator is hosted at erik-at-boson/open-asr-leaderboard-orze-ensemble. It runs the checkpoints sequentially and includes the sum of all three inference times in RTFx.

Self-evaluated public development results

These results use the evaluator and normalizer from the Open ASR Leaderboard repository at commit d1e99b25524814332d6868a5645e568670834cfb.

Aggregate component throughput is 29.66 RTFx, calculated from the sum of all three component inference times. The evaluator also records consensus alignment overhead when it creates fresh manifests.

Integrity recheck on 2026-08-23: the pinned scorer reproduced every displayed WER and the 3.81 macro mean; regenerating all seven manifests from the recorded anchor/voter manifests reproduced all 72,341 rows exactly (zero differing rows). This is a reproducibility statement, not an official verification.

Public proxy checks used during development

AppTek Call-Center Dialogues is a public CC-BY-SA-4.0 dataset and is unrelated to the leaderboard's private Appen datasets. Three deterministic 200-segment selections (seeds 8262026, 8262027, and 8262028) used different calls and had zero pairwise segment overlap. They were excluded from the Orze anchor's fine-tuning data. Lower WER is better.

Both individual voters were worse than the anchor on all three selections. This supports the behavior of the unanimity gate on these selected public proxies; it does not establish performance on the private leaderboard families.

A deterministic, balanced 300-utterance selection from a stable 20% MInDS-14 partition spanning en-AU, en-GB, and en-US was also used during development. This partition was excluded from anchor fine-tuning. Overall WER changed from 31.24 to 31.04, and locale-macro WER from 31.32 to 31.13. The consensus changed 87 normalized transcripts: 38 improved, 34 worsened, and 15 were error-neutral, for 10 fewer word errors. Hojo and MOSS individually scored 52.37 and 39.18 WER on this selection.

Scope and limitations

This is a 9.64B-parameter composite, using evaluator-reported component counts rounded to two decimals (2.04B Orze Qwen, 5.18B Hojo, and 2.42B MOSS). It trades throughput and total storage for accuracy. Components run sequentially, so peak inference memory is the maximum of the components rather than their sum. The public results were actively used in system selection and should not be interpreted as a blind-test comparison.

License

The consensus code and all three checkpoint repositories at the pinned revisions declare Apache-2.0. The public proxy datasets retain their own licenses; no proxy audio is redistributed in this repository.