InternVL3_5-1B_GPTQ_INT4

InternVL3_5-1B_GPTQ_INT4

InternVL3.5-1B (InternViT + Qwen3-0.6B) converted to run on the Axera NPU with w4a16 quantization, packaged in ax-llm model-directory format.

• Compatible with Pulsar2 5.1-patch1
• Context length: 2k, max prefill: 1k, image input: 448×448

This repository provides two self-contained ax-llm model directories, one per target chip:

Each directory is a complete ax-llm model dir:

InternVL3_5-1B_GPTQ_INT4-AX650/
├── config.json                    # ax-llm runtime config (vlm_type=InternVL3, tokenizer_type=InternVL3_5)
├── internvl3-5-1b_tokenizer.txt   # tokenizer
├── post_config.json               # sampling config
├── internvl3-5_axmodel/           # LLM: qwen3_p128_l0..l27 (28) + qwen3_post + embed
└── vit-models/                    # InternViT image encoder

Support Platform

• AX650
  • AX650N DEMO Board
  • M4N-Dock (爱芯派Pro)
  • M.2 Accelerator card (AXCL)
• AX620E
  • AX620E DEMO Board

How to use

Build/obtain the ax-llm runtime: AXera NPU LLM Runtime (ax-llm)

AX650N host or AXCL M.2 accelerator card — pick the -AX650 directory:

# interactive chat
./axllm run   InternVL3_5-1B_GPTQ_INT4-AX650

# OpenAI-compatible HTTP server (default port 8000)
./axllm serve InternVL3_5-1B_GPTQ_INT4-AX650

On AXCL, choose the card via AXLLM_DEVICES=<id> or the "devices" field in config.json.

AX620E board — pick the -AX620E directory:

./axllm run InternVL3_5-1B_GPTQ_INT4-AX620E

Source & conversion

• Base model: OpenGVLab/InternVL3_5-1B
• How to convert LLM/VLM from HuggingFace to axmodel (Pulsar2)
• License: BSD-3-Clause