North-Micro-Vision-Instruct-PARO

Jeethu/North-Micro-Vision-Instruct

Pairwise Rotation Quantization for Efficient Reasoning LLM Inference

<p> <a href="https://arxiv.org/abs/2511.10645"><img src="https://img.shields.io/badge/arXiv-2511.10645-b31b1b.svg" alt="Paper"></a> <a href="https://paroquant.z-lab.ai"><img src="https://img.shields.io/badge/Blog-ParoQuant-blue" alt="Blog"></a> <a href="https://huggingface.co/collections/z-lab/paroquant"><img src="https://img.shields.io/badge/%F0%9F%A4%97-Models-yellow" alt="Models"></a> <a href="https://pypi.org/project/paroquant/"><img src="https://img.shields.io/pypi/v/paroquant" alt="PyPI"></a> </p>

ParoQuant is the state-of-the-art INT4 quantization for LLMs. It closes the accuracy gap with FP16 while running at near-AWQ speed. Supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX). For more information, see https://github.com/z-lab/paroquant.

Jeethu/North-Micro-Vision-Instruct is a 4-bit CohereLabs/North-Micro-Vision-Instruct quantized with ParoQuant.

Evaluation

The following evaluations were run on 2026-08-14 with deterministic greedy decoding. The source checkpoint was evaluated in its native BF16 dtype. This ParoQuant checkpoint uses INT4 language projections (group size 128, krot=8) with FP16 retained tensors, including the vision encoder. For context, the published MLX affine 4-bit checkpoint uses group size 64 with BF16 retained tensors.

Vision

ChartQA uses a fixed seed-0 stratified sample of 50 human_test and 50 augmented_test examples. Scoring follows the VLMEvalKit relaxed rule: case-insensitive exact text matching or a 5% relative tolerance for numeric answers. MMStar uses the complete validation split and extracts the selected option from deterministic generations.

Against source BF16, ParoQuant changes ChartQA by 0.00 percentage points (paired bootstrap 95% CI: -3.00 to +3.00) and MMStar by -0.20 points (95% CI: -1.80 to +1.40). A targeted multi-image color-ordering, synthetic OCR, and object-counting smoke suite was also passed exactly by source BF16 and ParoQuant. MLX was semantically correct on all three cases and exact on two; its color response was verbose.

Text

WikiText-2 perplexity uses 64 non-overlapping sequences of 512 tokens from the test split. ARC-Challenge uses the complete labeled test split. HellaSwag uses a fixed seed-0 sample from the validation split. ARC-Challenge and HellaSwag are zero-shot greedy chat multiple-choice evaluations with identical prompts and tokenization across backends; they are not canonical lm-eval log-likelihood scores.

Against source BF16, the ParoQuant ARC-Challenge delta is -2.65 percentage points (paired bootstrap 95% CI: -4.18 to -1.11), while its HellaSwag delta is -1.30 points (95% CI: -2.80 to +0.20).

Dataset revisions were pinned to 9e63b7df1592a1c2158e735cc1725454aef0d6d9 (ChartQA), bc98d668301da7b14f648724866e57302778ab27 (MMStar), 210d026faf9955653af8916fad021475a3f00453 (ARC), 218ec52e09a7e7462a5400043bb9a69a41d06b76 (HellaSwag), and b08601e04326c79dfdd32d625aee71d232d685c3 (WikiText).