rwkv-vl-1.5v100m-finevisionmax

RWKV-VL 1.5B-v100M FineVisionMax

Evaluation

Evaluated with lmms-eval commit 8e2e7026, greedy decoding, and no dataset limit. TextVQA uses the 500-example textvqa_val_lite configuration; the other benchmarks use their complete test sets.

MME values are category-aggregated point totals rather than percentages.

Quick Start

The model uses custom code and Flash Linear Attention. Install a PyTorch build compatible with your CUDA environment, then install:

pip install "transformers==5.8.0" "accelerate==1.13.0" \
  "flash-linear-attention==0.5.0" pillow

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "ZoomFly/rwkv-vl-1.5v100m-finevisionmax"

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
    )

generated_ids = output_ids[:, inputs.input_ids.shape[1] :]
answer = processor.batch_decode(
    generated_ids,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)[0]
print(answer)

Model Configuration

Notes

• trust_remote_code=True is required. Review the bundled Python files before loading code from an untrusted source.
• The model may emit an empty <think>...</think> block before its final answer.
• This is a research checkpoint and can make visual-reading, counting, and arithmetic errors.
• AI2D is an in-domain benchmark for this checkpoint. TextVQA Val Lite is not comparable to evaluations on the full TextVQA validation set.
• The publisher should review and add the appropriate model and data licenses before public release.