StreamOPD-4B-ST-CueGate

StreamOPD-4B-ST-CueGate

A 4B streaming video-understanding model, post-trained from Qwen3.5-4B with on-policy distillation from a Qwen3.5-9B teacher under ST-CueGate, the spatio-temporal cue gating method from StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding.

• Paper: https://arxiv.org/abs/2608.16320
• Code: https://github.com/UniX-AI-Lab/StreamOPD
• Project page: https://unix-ai-lab.github.io/StreamOPD/

What is different about this model

The model is trained for a deliberately austere streaming setting: at answer time it sees only the 4 most recent frames at 1 fps — no memory bank, no retrieval, no KV-cache compression, and no generated reasoning trace. All of the capability lives in the weights rather than in test-time machinery.

Training uses on-policy distillation in thinking mode, while deployment is in instruct mode. During training the frozen teacher scores the student's own response twice — once conditioned on a grounded spatio-temporal cue and once without it — and the resulting per-token likelihood ratio is aggregated into a response-level weight that gates the distillation advantage. The cue is a training-time signal only: it is never present at inference, so the deployment path is identical to a plain Qwen3.5-4B.

Results

Evaluated in instruct mode with greedy decoding under the recent-4-frame protocol. StreamingBench and OVO-Bench use recent-4 frames at 1 fps; Video-MME and LongVideoBench use their standard protocols with at most 32 frames.

The full comparison, including per-subtask OVO-Bench breakdowns, the teacher-conditioning ablations, and the self-distillation variant, is in the paper.

Usage

from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "UniX-Lab/StreamOPD-4B-ST-CueGate"
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id, torch_dtype="bfloat16", device_map="auto"
).eval()
processor = AutoProcessor.from_pretrained(model_id)

messages = [{
    "role": "user",
    "content": [
        {"type": "video", "video": "clip.mp4"},
        {"type": "text", "text": "What did the person pick up?\nA. book  B. cup  C. apple\n"
                                 "Only give the best option's letter directly."},
    ],
}]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

out = model.generate(**inputs, max_new_tokens=32, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Run with thinking disabled — the model is trained to answer directly, and the reported scores use instruct-mode greedy decoding. Set FORCE_QWENVL_VIDEO_READER=decord for video decoding.

To reproduce the streaming evaluation exactly, use the evaluators in the code repository, which implement the recent-window protocol and the OVO-Bench B+R macro:

bash scripts/eval/run_all.sh <path-to-this-model> my_run 0,1,2,3
bash scripts/eval/score_all.sh my_run

Training

Training data is derived from public sources (LLaVA-Video-178K and Kinetics-700 clips); the data pipeline, the parquets, and the cue-generation tooling are in the code repository.

Limitations

• Answers come from a 4-frame window, so questions needing evidence outside that window are out of scope by construction.
• Distillation from a larger teacher reduces the willingness to abstain on unanswerable queries (OVO-Bench's HLD subtask) relative to the untrained base model. The paper reports a self-distillation variant that recovers this.
• Trained and evaluated on English data.

Citation

@article{wu2026streamopd,
  title   = {StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating
             for Streaming Video Understanding},
  author  = {Wu, Keming and Wang, Baoyi and Zhang, Kaichen and An, Xiang and
             Yang, Zuhao and Wang, Sudong and Zhu, Haowei and Huang, Tingxuan and
             Gao, Hongcheng and Wang, Bin},
  journal = {arXiv preprint arXiv:2608.16320},
  year    = {2026}
}