whisper-large-v3-turbo-tokenizer-prevfix

whisper-large-v3-turbo tokenizer with <|startofprev|> renamed to <|prev|>

A drop-in tokenizer-only workaround for vllm-project/vllm#53206: passing the OpenAI-compatible prompt field to vLLM's /v1/audio/transcriptions on a Whisper model corrupts decoding — degraded output on short audio, and on audio longer than 30 s the response silently collapses to only the final chunk's text.

No model weights here. These are the official openai/whisper-large-v3-turbo tokenizer files with exactly one change: the string form of token id 50362 is renamed from <|startofprev|> to <|prev|> (applied consistently in tokenizer.json, tokenizer_config.json, added_tokens.json and special_tokens_map.json). Vocabulary size and every id are unchanged.

The repo also carries the unmodified official config.json: vLLM resolves a remote --tokenizer repo through AutoConfig and refuses to start when the repo lacks a config.json with a model_type key (local directory mounts skip that check). No weights are read from this repo.

Why this works

vLLM builds the Whisper decoder prompt as:

decoder_text = (
    f"<|prev|>{request_prompt}" if request_prompt else ""
) + f"<|startoftranscript|><|{language}|><|{task_type}|><|notimestamps|>"

<|prev|> is not in the Whisper vocabulary, so the stock tokenizer splits it into six ordinary text tokens (<, |, pre, v, |, >) placed in front of <|startoftranscript|>, which corrupts decoder conditioning. The real prompt-conditioning token is <|startofprev|> (id 50362).

With this tokenizer mounted, the same <|prev|> string tokenizes straight to id 50362 — the genuine sot_prev embedding — so the prompt field behaves as intended again, with zero code changes to vLLM. Requests without a prompt are byte-for-byte unaffected.

Usage

vllm serve openai/whisper-large-v3-turbo \
  --tokenizer ardge-hf/whisper-large-v3-turbo-tokenizer-prevfix

Then the standard OpenAI prompt field works:

curl http://localhost:8000/v1/audio/transcriptions \
  -F [email protected] \
  -F response_format=verbose_json \
  --form-string 'prompt=這是繁體中文的語音轉錄。'

Measured on vllm/vllm-openai:nightly (v0.26.1rc1, 2026-08): without the fix the request above returns a single hallucinated segment; with it, the full transcript comes back and the prompt bias takes effect (e.g. Mandarin output switches from Simplified to Traditional script with punctuation).

Verify

from transformers import AutoTokenizer

t = AutoTokenizer.from_pretrained("ardge-hf/whisper-large-v3-turbo-tokenizer-prevfix")
assert t.encode("<|prev|>", add_special_tokens=False) == [50362]
assert len(t) == 51866  # no phantom tokens added

When to stop using this

The proper fix — vLLM emitting <|startofprev|> in the first place — is vllm-project/vllm#53207. Once your vLLM build contains it, drop the --tokenizer flag and go back to the stock tokenizer.

Scope and caveats

• Built for openai/whisper-large-v3-turbo and valid for the v3 family, where sot_prev is id 50362. The v2 family (large-v2, small, …) uses id 50361 — regenerate per checkpoint rather than reusing this one.
• Client-side workarounds do not exist for this bug: the junk <|prev|> text tokens are prepended server-side before any client-controlled content, and even embedding a literal <|startofprev|> inside the prompt value still collapses (measured). The tokenizer has to be swapped server-side.
• Decoding id 50362 now renders <|prev|>; with skip_special_tokens=True (the transcription default) this is never visible.
• Serving/inference workaround only — do not fine-tune against this tokenizer.

Provenance

Tokenizer files come from openai/whisper-large-v3-turbo (Apache-2.0), modified as described above. The rename was verified against the official added_tokens_decoder tables of whisper-large-v3-turbo and whisper-large-v3, and the reference specials list in openai/whisper tokenizer.py.