Downloads · 30 days
40
4% of all-time downloads
Kimi-VL-A3B-Thinking-2506-W4A16
88plug/Kimi-VL-A3B-Thinking-2506-W4A16
Kimi-VL-A3B-Thinking-2506-W4A16 is a image-text-to-text model from 88plug. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
W4A16 post-training quantization of moonshotai/Kimi-VL-A3B-Thinking-2506. Language-model Linear layers quantized; vision tower and merger kept BF16.
All-time downloads
977
Public
Parameters
16.9B
18.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors18.5 GB · 100%
Parameter types
How the weights are stored.
I815.3B · 91%
Try a prompt
Base models
Model card
From the Hugging Face model README
Kimi-VL-A3B-Thinking-2506-W4A16
W4A16 post-training quantization of moonshotai/Kimi-VL-A3B-Thinking-2506. Language-model Linear layers quantized; vision tower and merger kept BF16.
At a Glance
| Property | Value |
|---|---|
| Base model | moonshotai/Kimi-VL-A3B-Thinking-2506 |
| Release tier | Provisional (datafree RTN — re-quant scheduled) |
| Quant method | datafree RTN W4A16 (AutoRound blocked — KNOWN-FAILURES) |
| FLAC status | Not measured (T+7d milestone) |
| Quant format | compressed-tensors (Marlin G32) |
| Disk size | ~18 GB |
Quick Start
docker run --gpus device=0 -p 8080:8080 \
vllm/vllm-openai:v0.21.0-cu129-ubuntu2404 vllm serve \
88plug/Kimi-VL-A3B-Thinking-2506-W4A16 \
--trust-remote-code \
--max-model-len 8192 \
--gpu-memory-utilization 0.90
Benchmarks
| Metric | Status |
|---|---|
| Throughput (tok/s) | In progress — T+7d milestone |
| MMLU delta vs BF16 | In progress — T+7d milestone |
| RULER@128k | In progress — T+30d milestone |
No fabricated numbers. Results will be published to this card when measured.
License
MIT (per base model).
About
88plug AI Lab ships compressed-tensors quantizations for native vLLM v0.21.0+ deployment.
This release: Provisional tier — datafree RTN (weight-only rounding, no calibration corpus). A gold AutoRound re-quant is scheduled; 88plug architecture forbids new provisional W4A16 uploads.
Browse all releases → huggingface.co/88plug
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- kimi_vl
- License
- mit
- Languages
- en, zh
- Created
- Jul 1, 2026
- Updated
- Jul 10, 2026