OPOD-Qwen2.5-Omni-3B

OPOD-Qwen2.5-Omni-3B

OPOD (On-Policy Omni Distillation) consolidates separate text, image, and audio teachers into a single omni-modal student. This model is the 3B student from the paper OPOD: On-Policy Omni Distillation, post-trained from Qwen/Qwen2.5-Omni-3B.

It reaches an overall average of 46.2 across twelve text / vision / audio / omni-modal benchmarks — +5.5 over the base model (the largest base-relative gain of the three scales) and +1.7 over the strongest comparator. The teachers are discarded after training, so inference uses one model with no teacher-ensemble latency or memory.

• 📄 Paper: arXiv:2607.20918
• 💻 Code: github.com/VincentZhao2002/OPOD
• 🤗 Other scales: OPOD-Qwen3-Omni-30B-A3B · OPOD-Qwen2.5-Omni-7B

Method

Post-training on pooled multimodal data often fails to preserve the strengths of modality specialists, and standard on-policy distillation does not extend cleanly to several teachers: their guidance conflicts on a shared backbone, and symmetric distribution matching imposes a teacher ceiling. OPOD addresses this with three components:

Each student rollout is routed by its input modality to the matching teacher. Teachers are Qwen3-Omni modality specialists, so this model also demonstrates that OPOD transfers across model families and tokenizers (MoE teachers → dense Qwen2.5-Omni student).

Results

Accuracy (%) on the paper's twelve benchmarks. Bold = best in the block.

Highlights:

• +5.5 over base, +3.8 over pooled GRPO, +1.7 over the strongest comparator (ExOPD).
• Ranks 1st or 2nd on 9 of 12 benchmarks.
• Largest base-relative gains: MMLU-Pro +18.7 and ChartQA +10.2.
• The method ordering is identical at 3B, 7B, and 30B — the gain is not tied to one parameter regime.

Evaluation protocol. Accuracy, temperature 0.7, top-p 0.7, max prompt / response length 8,192 tokens, identical for all methods. Training prompts are disjoint from every benchmark (exact and normalized-string overlaps removed).

Usage

This model keeps the Qwen2.5-Omni architecture and processor, so it is a drop-in replacement for the base model.

Prompt format (important)

The model was RL-trained to reason inside <think></think> and put the final answer inside <answer></answer>. Use a system prompt of the same form, otherwise accuracy will drop and answers become harder to parse. The prompts used in the paper are in eval/prompts/system_prompts.py.

For image / audio / omni multiple-choice questions:

Please think about this question as if you were a human pondering deeply, carefully
considering BOTH the visual and the audio information before answering, engaging in an
internal dialogue using expressions such as let me think, wait, hmm, oh I see, or let's
break it down, including self-reflection or verification in the reasoning process,
providing the detailed reasoning between the <think> </think> tags, and finally giving
only the single option letter (e.g., A, B, C, D, etc.) as the final answer within the
<answer> </answer> tags.

For text problems:

You are a careful problem solver. Think step by step inside <think> </think> tags, then
output the final answer in the exact format requested at the end of the user prompt (do
not add any extra wrappers around the final answer).

transformers

from transformers import Qwen2_5OmniForConditionalGeneration, Qwen2_5OmniProcessor

model_id = "Tung111/OPOD-Qwen2.5-Omni-3B"
model = Qwen2_5OmniForConditionalGeneration.from_pretrained(
    model_id,
    dtype="auto",
    device_map="auto",
    enable_audio_output=False,   # only the thinker was post-trained; see Limitations
)
processor = Qwen2_5OmniProcessor.from_pretrained(model_id)

SYSTEM = (
    "You are a careful problem solver. Think step by step inside <think> </think> tags, "
    "then output the final answer in the exact format requested at the end of the user "
    "prompt (do not add any extra wrappers around the final answer)."
)

conversation = [
    {"role": "system", "content": [{"type": "text", "text": SYSTEM}]},
    {"role": "user", "content": [
        {"type": "image", "image": "https://example.com/chart.png"},
        {"type": "text", "text": "What is the highest value in the chart? "
                                 "Put the final answer in <answer></answer>."},
    ]},
]

text = processor.apply_chat_template(conversation, add_generation_prompt=True, tokenize=False)
inputs = processor(text=text, return_tensors="pt").to(model.device)

ids = model.generate(**inputs, max_new_tokens=8192, temperature=0.7, top_p=0.7,
                     do_sample=True, return_audio=False)
print(processor.batch_decode(ids[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])

On older transformers releases the class is named Qwen2_5OmniModel. Use the processor's process_mm_info helper for audio and video inputs, exactly as with the base model.

vLLM

vllm serve Tung111/OPOD-Qwen2.5-Omni-3B \
  --max-model-len 16384 --allowed-local-media-path / --trust-remote-code

Then query the OpenAI-compatible endpoint with the system prompts above.

Training details

Limitations

• Post-training targets the thinker (text/reasoning) pathway. The talker / speech-generation components are inherited unchanged from the base model and were not optimized or evaluated — load with enable_audio_output=False for reasoning use cases.
• Optimized for verifiable reasoning with the <think> / <answer> format; free-form chat, long-form generation, and speech output are outside the evaluated scope.
• Absolute scores on competition math (AIME) remain low at this scale.
• Single run per configuration with a fixed seed; no multi-seed variance is reported.
• Inherits the base model's biases and knowledge cutoff. Not evaluated for safety alignment.

Citation

@article{zhao2026opod,
  title   = {OPOD: On-Policy Omni Distillation},
  author  = {Zhao, Tong and Hu, Yuyang and Zhu, Yutao and Li, Reed and
             Liang, Haijin and Shi, Haibo and Lu, Yu and Dou, Zhicheng},
  journal = {arXiv preprint arXiv:2607.20918},
  year    = {2026},
  url     = {https://arxiv.org/abs/2607.20918}
}

License

Derived from Qwen2.5-Omni-3B, which is released under the Qwen Research License. This model inherits those terms — see the base model license.