kanana-1.5-v-3b-instruct

<p align="center"> <br> <picture> <img src="./assets/logo/kanana-logo.png" width="60%" style="margin: 40px auto;"> </picture> </br> <p align="center"> 🤗 <a href="https://kko.kakao.com/kananallm">1.5 HF Models</a> &nbsp | &nbsp 📕 <a href="https://tech.kakao.com/posts/714">Blog</a> &nbsp <br>

Table of Contents

• Kanana-1.5-v-3b-instruct
• Intended Use
• Model Details
• Evaluation
  • Model Configuration Summary
  • Overview
  • Image Benchmarks (EN)
  • Image Benchmarks (KO)
  • Multimodal Instruction Following Benchmarks (EN, KO)
  • Note on Benchmarking Methodology
• Usage
  • Requirements
  • Quickstart
• Limitations
• Contributors
• Contact

<br>

kanana-1.5-v-3b-instruct

The Unified Foundation Model (UFO) task force of Kanana at Kakao developed and released the Kanana-V family of multimodal large language models (MLLMs), a collection of pretrained text/image-to-text (TI2T) models.

Intended Use

kanana-1.5-v-3b-instruct is intended for research and application development in multimodal understanding and text generation tasks. Typical use cases include image captioning, document understanding, OCR-based reasoning, and multimodal instruction following in both English and Korean. The model is optimized for both general-purpose and Korea-specific benchmarks, making it suitable for bilingual environments.

Model Details

• Developed by: Unified Foundation Model (UFO) TF at Kakao
• Language(s) : ['en', 'ko']
• Model Architecture: kanana-1.5-v-3b-instruct has 3.6B parameters and contains image encoder, C-abstractor, and kanana-1.5-3b-instruct language model.
• Input: The models accept text and image inputs.
• Output: The models generate text only.
• Context Length: 32k
• Knowledge Cutoff Date: June 30, 2024
• Model Release Date: Jul 24, 2025.
• License: kanana-license

Evaluation

Model Configuration Summary

Overview

Image Benchmarks (EN)

Image Benchmarks (KO)

Multimodal Instruction Following Benchmarks (EN, KO)

Note on Benchmarking Methodology

All benchmarks were re-measured under identical software conditions to ensure fair comparison.

• VLMEvalKit was used for MMMU, MathVista, ScienceQA, MIA-Bench, MM-IFEval and MM-OmniAlign.

• lmms-eval was employed for DocVQA, ChartQA, OCRBench, InfoVQA, TextVQA, RealWorldQA, MMStar, MMB, and SEED-image.

• HCX-SEED-Vision-3B was evaluated without the use of any auxiliary tools (e.g., external OCR engines or Lens features), as such tools are not publicly available and therefore not included in our evaluation setup.

• Important note for ChartQA: It was identified that the original rule-based parser used by lmms-eval marked answers ending with a period (".") as incorrect due to parsing issues. To address this, the parser logic was modified to remove any trailing period before parsing the response. All ChartQA evaluations presented here reflect results obtained after applying this parser adjustment.

The following in-house benchmarks evaluate Korean-language tasks and Korea-specific knowledge:

Usage

Requirements

pip install transformers accelerate timm omegaconf

transformers>=4.45.0 or the latest version is recommended.

Quickstart

The following is a code snippet that briefly demonstrates how to load a model and process input data using the AutoClass from transformers.

from PIL import Image
import torch
from transformers import AutoModelForVision2Seq, AutoProcessor

MODEL = "kakaocorp/kanana-1.5-v-3b-instruct"

# Load the model on the available device(s)
model = AutoModelForVision2Seq.from_pretrained(
    MODEL,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)
model.eval()

# Load processor
processor = AutoProcessor.from_pretrained(MODEL, trust_remote_code=True)

# Prepare input batch
batch = []
for _ in range(1):  # dummy loop to demonstrate batch processing
    image_files = [
        "./examples/waybill.png"
    ]

    sample = {
        "image": [Image.open(image_file_path).convert("RGB") for image_file_path in image_files],
        "conv": [
            {"role": "user", "content": " ".join(["<image>"] * len(image_files))},
            {"role": "user", "content": "사진에서 보내는 사람과 받는 사람 정보를 json 형태로 정리해줘."},
        ]
    }

    batch.append(sample)
    
inputs = processor.batch_encode_collate(
    batch, padding_side="left", add_generation_prompt=True, max_length=8192
)
inputs = {k: v.to(model.device) if isinstance(v, torch.Tensor) else v for k, v in inputs.items()}

# Set the generation config
gen_kwargs = {
    "max_new_tokens": 2048,
    "temperature": 0,
    "top_p": 1.0,
    "num_beams": 1,
    "do_sample": False,
}

# Generate text
gens = model.generate(
    **inputs,
    **gen_kwargs,
)
text_outputs = processor.tokenizer.batch_decode(gens, skip_special_tokens=True)
print(text_outputs)  # ['```json\n{\n  "보내는분": {\n    "성명": "카카오",\n    "주소": "경기도 성남시 판교역로 166"\n  },\n  "받는분": {\n    "성명": "카나나",\n    "주소": "제주도 제주시 첨단로 242"\n  }\n}\n```']

Limitations

• The model may generate inaccurate or misleading content, especially in scenarios requiring precise factual understanding (e.g., scientific diagrams or mathematical reasoning).
• Performance on languages other than Korean and English has not been evaluated and may be poor.
• The model is not designed for medical, legal, or other high-stakes domains.
• The model may reflect social biases present in the pretraining data.

Contributors

• Beomhee Park, Byeonguk Bae, Byungseok Roh, Daejin Jo, Donghee Son, Dongjin Lee, Hyunwoong Ko, Jaemyung Lee, Jeehye Lee, Sunghun Kang, Wooyoung Kang
• Listed in alphabetical order (first name)

Contact

• Kanana MLLM Core Team Technical Support: [email protected]
• Business & Partnership Contact: [email protected]