HiViS-llava-v1.6-vicuna-7b-hf

HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models

Zhinan Xie, Peisong Wang, Shuang Qiu, Jian Cheng

<a href="https://arxiv.org/abs/2509.23928"><img src="https://img.shields.io/static/v1?label=arXiv&message=Paper&color=red&logo=arxiv"></a> <a href="https://github.com/lnn-ops/HiViS"><img src="https://img.shields.io/static/v1?label=GitHub&message=Code&color=black&logo=github"></a>

Model Description

This repository contains the HiViS drafter checkpoint for llava-hf/llava-v1.6-vicuna-7b-hf.

This checkpoint is not a standalone vision-language model. It must be used with the corresponding base model and the HiViS inference code.

Overview

Speculative decoding has proven effective for accelerating inference in Large Language Models (LLMs), yet its extension to Vision-Language Models (VLMs) remains limited by the computational burden and semantic inconsistency introduced by visual tokens. Recent studies reveal that visual tokens in large VLMs are highly redundant, and most of them can be removed without compromising generation quality. Motivated by this observation, we propose HiViS (Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models), a framework that utilizes the target VLM as a semantic fusion model, allowing the drafter to obtain visual information without explicitly processing visual tokens, ensuring that the drafter's prefill sequence length matches that of the textual tokens. Furthermore, HiViS employs a time-step-aware aligned training scheme that allows the drafter to autonomously propagate and refine instructive visual-textual semantics during independent drafting, guided by step-dependent bias-correction residuals. Extensive experiments across representative VLMs and benchmarks demonstrate that HiViS achieves significant improvements in average acceptance length and speedup ratio.

Usage

python -m hivis.evaluation.ge_hivis_answer \
  --draft-method hivis \
  --base-model-path llava-hf/llava-v1.6-vicuna-7b-hf \
  --ea-model-path <path_or_huggingface_repo_for_this_checkpoint> \
  --dataset ChartQA \
  --answer-file outputs/chartqa_hivis.jsonl

The drafter checkpoint must match the exact base-model architecture and size.

Files

• config.json: configuration of the HiViS drafter.
• model.safetensors: HiViS drafter weights.

Citation

@inproceedings{xie2026hivis,
  title={Hivis: Hiding visual tokens from the drafter for speculative decoding in vision-language models},
  author={Xie, Zhinan and Wang, Peisong and Qiu, Shuang and Cheng, Jian},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={8952--8961},
  year={2026}
}

License

This checkpoint follows the license of its corresponding base model. Please also follow the license of the HiViS code repository.