Downloads · 30 days
23
11% of all-time downloads
qwen3.5-4b-awq-v128k
Teora/qwen3.5-4b-awq-v128k
qwen3.5-4b-awq-v128k is a machine learning model from Teora. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Quantized & vocab-pruned Qwen3.5-4B used by team AFM-6j6duhm6 to place 2nd (7.708×) in the AdaptFM "Efficient Qwen" Challenge (ICML 2026).
All-time downloads
215
Public
Parameters
4.4B
5.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.5 GB · 100%
Parameter types
How the weights are stored.
I322.2B · 50%
Base models
Model card
From the Hugging Face model README
Qwen3.5-4B — AWQ-INT4 (MLP) + v128k vocab pruning
Quantized & vocab-pruned Qwen3.5-4B used by team AFM-6j6duhm6 to place 2nd (7.708×) in the AdaptFM "Efficient Qwen" Challenge (ICML 2026).
Pair this checkpoint with the reproducible serving pipeline (suffix always-spec decoding, K=20, factor 3.0 + force-answer wrapper): 👉 https://github.com/Teora/efficient-qwen-suffix-spec
What's modified vs the base model
| Change | Detail |
|---|---|
| Quantization | AWQ-INT4 on MLP only (attention, layer 0, MTP head, vision tower stay FP16). awq_marlin kernel. |
| Vocab pruning | Tied embed / lm_head pruned 248,320 → 131,101 tokens: keep contiguous IDs [0,131072) + all special/added tokens (remapped contiguously). This reduces parameters. |
| MTP head | Kept intact (native multi-token-prediction head preserved). |
Vocab pruning shrinks the BF16 tied embed/lm_head (~1.27 GB, the dominant per-token decode memory cost) by ~1.9×, speeding decode/verify while preserving the GPQA-Diamond (thinking) quality gate. v96k pruning failed GPQA; v128k (131k) is the minimum vocab that preserves it.
Quality (competition gates, all PASS)
| Benchmark | Threshold | Result |
|---|---|---|
| MMLU-Pro | ≥ 0.621 | PASS |
| IFEval | ≥ 0.814 | PASS |
| GPQA-Diamond (thinking) | ≥ 0.630 | PASS |
Usage
# 1) download this checkpoint into ./qwen-awq-v128k/
huggingface-cli download Teora/qwen3.5-4b-awq-v128k --local-dir qwen-awq-v128k
# 2) build & serve via the pipeline repo
git clone https://github.com/Teora/efficient-qwen-suffix-spec && cd efficient-qwen-suffix-spec
docker build -f Dockerfile -t efficient-qwen-2nd:latest .
docker run -d --gpus '"device=0"' -p 8080:8080 efficient-qwen-2nd:latest
License
Apache-2.0 (Qwen3.5-4B is Apache-2.0; this derivative checkpoint inherits it).
Citation
Team AFM-6j6duhm6 · Minjae Park (CJ Corporation) · AdaptFM Efficient Qwen Challenge, ICML 2026.
- Type
- qwen3_5
- License
- apache-2.0
- Languages
- en
- Created
- Jun 30, 2026
- Updated
- Jun 30, 2026