Qwen3.8-27B-FP8-Pessoa

Qwen3.8-27B-FP8-Pessoa

FP8 (W8A16) weight-only, GPTQ-quantized variant of Qwen/Qwen3.8-27B, produced with llm-compressor and saved in compressed-tensors format for direct use with vLLM.

Pessoa is the best FP8 quantization of Qwen3.8-27B of a set of experiments with autoresearch it reached the the lowest KLD (0.0056) achieved by any practical-size FP8 model in a 97-model quantization sweep. Every linear language-model layer MLP, self-attention, and linear-attention uses FP8 E4M3 weight-only (W8A16) with per-channel weights, BF16 activations, and GPTQ Hessian error correction. The vision tower and MTP head are kept in BF16, so the model retains full image/video understanding and multi-token-prediction speculative decoding capability.

Named after Fernando Pessoa (1888–1935), the Portuguese poet who wrote under dozens of heteronyms — each a distinct literary persona with its own voice. This model too has multiple selves: the quantized language model, the unquantized vision tower, and the MTP speculative head three complete personas in one checkpoint.

---

Model Details

Qwen3.8-27B (qwen3_5 / Qwen3_5ForConditionalGeneration):

• Type: Vision-language model (hybrid linear + full attention language model + vision encoder + MTP head)
• Parameters: 27B (language model) + 460.7M (vision tower) + 82.3M (MTP head)
• Layers: 64 — 48 linear-attention (GatedDeltaNet) + 16 full-attention (every 4th layer: 0, 4, 8, …, 60)
• Hidden size: 5,120
• Intermediate size: 17,408 (MLP is 3.4× hidden — dominates parameter count)
• Attention heads: 24 × 256 dim
• KV heads: 4 (GQA 6:1) — only on 16 full-attention layers
• Vocabulary: 248,320
• Activation: SiLU
• Context: 262,144 tokens native
• Vision encoder: 27-layer ViT, hidden size 1,152, patch size 16, 16 heads, temporal patch size 2
• MTP: 1 hidden layer, multi-token-prediction speculative decoding head

Each linear-attention layer has in_proj_qkv, in_proj_z, out_proj plus recurrent state paths in_proj_a/in_proj_b. Each full-attention layer has standard q/k/v/o_proj. All layers share gate_proj/up_proj/down_proj MLP blocks. The vision tower (model.visual.*) processes image/video patches and injects embeddings into the language model via image/video token boundaries. The MTP head (mtp.*) provides speculative decoding tokens for inference acceleration.

---

Quantization Configuration

Precision Assignment

Key difference from W8A8: This model uses weight-only FP8 (W8A16) — weights are FP8 E4M3 per-channel, but activations remain BF16. This eliminates activation quantization error entirely, halving KLD versus W8A8 (0.0056 vs 0.0102). The tradeoff is ~18% slower prefill (4,875 vs 6,213 tok/s) because BF16 activations require dequantization before GEMM.

GPTQ: Hessian-based weight correction, sequential targets Qwen3_5DecoderLayer, actorder=static, dampening_frac=0.01, block_size=128. GPTQ corrects the weight quantization residual by adjusting remaining weights using the calibration data's second-order statistics — this is what gives Pessoa 2.3× lower KLD than ModelOpt's max-calibration FP8 (0.0056 vs 0.0127).

Tensor Inventory

Modules Kept in BF16

Calibration

• Source: malaiwah/qwen38-27b-fidelity-suite-v3 tokens (181 contexts × 2048 tokens)
• Samples: 181
• Max sequence length: 2048
• Format: Decoded token IDs from suite, not raw text
• Dampening: dampening_frac=0.01 (default — confirmed optimal for FP8; tested 0.001–0.1, all within noise)

Compressed-Tensors Config

{
  "quant_method": "compressed-tensors",
  "format": "float-quantized",
  "quantization_status": "compressed",
  "version": "0.18.0",
  "config_groups": {
    "group_0": {
      "targets": ["Linear"],
      "weights": {
        "num_bits": 8,
        "type": "float",
        "symmetric": true,
        "group_size": null,
        "strategy": "channel",
        "dynamic": false,
        "actorder": "static",
        "observer": "memoryless_minmax"
      },
      "input_activations": null
    }
  },
  "kv_cache_scheme": {
    "num_bits": 8,
    "type": "float",
    "symmetric": true,
    "strategy": "tensor",
    "dynamic": false,
    "observer": "static_minmax"
  }
}

---

Evaluation

Distribution Fidelity (KLD)

Hidden-state capture after final RMSNorm → shared BF16 LM-head replay → full-vocabulary KL(BF16_ref ‖ candidate), two-pass log-sum-exp normalization. Body-only comparison (candidate LM head unused). Suite: 136 contexts × 2048 tokens, 278,392 scored positions over 248,320-token vocabulary.

KLD by Domain

Literary text remains the dominant KLD contributor — 5.1× the scientific stratum. Creative/prose generation is disproportionately affected by quantization, but the gap is narrower than in W8A8 configs (v17: 5.4×) and much narrower than 4-bit configs. Pessoa's weight-only approach preserves activation distributions, keeping literary KLD under 0.01.

Throughput

Single NVIDIA RTX PRO 6000 Blackwell (SM120, 96 GB), vLLM 0.26.1rc0, FlashInfer backend. Duration-based sustained decode, 30s cells, temperature 0.0, ignore_eos=true.

NOTE: This was not optimized for speed, but for equal grounds between all experiments and models tested. An optimized setup with a 6kpro reaches >100tok/s with MTP.

Composite Score

$$\text{score} = \frac{\text{prefill}{2k} \times \text{decode}{c1}}{\text{KLD} \times 10^6} = \frac{4875 \times 45.6}{0.00556 \times 10^6} = \mathbf{40.02}$$

Highest composite score among all practical-size FP8 models in the 97-model sweep.

---

Context Within Experiment Series

Pessoa is the FP8 weight-only winner from a systematic 97-model quantization sweep of Qwen3.8-27B, exploring NVFP4, MXFP8, and FP8 formats via both ModelOpt (NVIDIA) and GPTQ (llm-compressor). The sweep evaluated GPTQ vs ModelOpt, weight-only vs weight+activation, single-format vs mixed-precision, and per-layer sensitivity.

Key Finding: Weight-Only Beats Weight+Activation for FP8

Weight-only FP8 eliminates activation quantization error entirely. GPTQ corrects the weight quantization residual. The result is 2× better KLD than W8A8 and 2.3× better than ModelOpt at the cost of ~18% slower prefill (BF16 dequantization before GEMM).

Key Finding: GPTQ Beats ModelOpt by 2.3×

GPTQ's Hessian-based weight correction compensates for quantization error by adjusting remaining weights using calibration data's second-order statistics. ModelOpt's max calibration only computes scale factors without weight correction.

Pareto Position

Pessoa dominates the practical-size Pareto frontier: 2× lower KLD than any other ~28 GB model, with the highest composite score in its class.

---

Usage

# vLLM serve (requires Blackwell SM120 + FlashInfer)
vllm serve /models/Qwen3.8-27B-FP8-Pessoa \
  --max-model-len 262144 \
  --max-num-seqs 16 \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8 \
  --trust-remote-code \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

--max-num-seqs matters on this architecture. 48 of 64 layers use linear attention, and vLLM allocates one Mamba-style cache block per decode sequence. The default max_num_seqs=1024 can exceed available blocks and fail during CUDA graph capture with max_num_seqs (1024) exceeds available Mamba cache blocks. Lower to 16–512 or raise --gpu-memory-utilization. This is a property of the base model, not of quantization.

from vllm import LLM, SamplingParams

llm = LLM(model="/models/Qwen3.8-27B-FP8-Pessoa",
          quantization="compressed-tensors",
          kv_cache_dtype="fp8",
          max_model_len=262144,
          trust_remote_code=True)

# Text-only generation
out = llm.generate(
    ["Explain 4-bit quantization in two sentences."],
    SamplingParams(temperature=0.7, max_tokens=256),
)
print(out[0].outputs[0].text)

# Vision: image + text via chat template
from vllm.inputs import TextPrompt
messages = [{
    "role": "user",
    "content": [
        {"type": "image_url", "image_url": {"url": "https://example.com/image.png"}},
        {"type": "text", "text": "Describe this image."},
    ],
}]
prompt = llm.get_tokenizer().apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True)
out = llm.generate(
    {"prompt": prompt, "multi_modal_data": {"image": image_placeholder}},
    SamplingParams(temperature=0.7, max_tokens=512),
)
print(out[0].outputs[0].text)

Docker Compose

A docker-compose.yml is provided in the experiment workspace for one-command deployment. The image bundles the vLLM infernal-invocation fork with FlashInfer and SM120 kernels; only the quantized checkpoint must be mounted from the host.

# Place the checkpoint at ./models/Qwen3.8-27B-FP8-Pessoa, then:
docker compose up -d

# Or override the model path and port:
MODEL_DIR=/data/models/Qwen3.8-27B-FP8-Pessoa PORT=8001 docker compose up -d

The endpoint is OpenAI-compatible: http://localhost:8000/v1.

<details> <summary><b>docker-compose.yml</b></summary>

# docker-compose.yml — Qwen3.8-27B-FP8-Pessoa
#
# Serves the FP8 W8A16 weight-only GPTQ quantization via mainline vLLM.
# Image: vllm/vllm-openai:latest
#
# Requirements:
#   - NVIDIA Blackwell (SM120) GPU with FP8 E4M3 tensor cores
#   - NVIDIA Container Toolkit (nvidia-docker) installed on host
#   - Quantized checkpoint at ${MODEL_DIR}
#
# Customize via .env file or shell environment variables:
#
#   PORT=8000                           API server port
#   SERVED_MODEL_NAME=qwen38-27b-pessoa Model name returned by /v1/models
#   MAX_MODEL_LEN=262144                Maximum context length (tokens)
#   MAX_NUM_SEQS=16                     Maximum concurrent decode sequences
#   MAX_NUM_BATCHED_TOKENS=16384        Maximum tokens per batching iteration
#   GPU_MEMORY_UTILIZATION=0.97         Fraction of GPU memory for vLLM (0.0–1.0)
#   TENSOR_PARALLEL_SIZE=2              Tensor-parallel degree (1, 2, or 4)
#   KV_CACHE_DTYPE=fp8                  KV cache dtype: fp8 | auto | bfloat16
#   CUDA_VISIBLE_DEVICES=0,1            GPU indices to use
#   MODEL_DIR=./models/Qwen3.8-27B-FP8-Pessoa  Host path to checkpoint

services:
  qwen38-27b-pessoa:
    image: vllm/vllm-openai:latest
    container_name: qwen38-27b-pessoa
    ipc: host
    ports:
      - "${PORT:-8000}:${PORT:-8000}"
    volumes:
      - ${MODEL_DIR:-./models/Qwen3.8-27B-FP8-Pessoa}:/models/Qwen3.8-27B-FP8-Pessoa:ro
      - vllm-cache:/cache
      - hf-cache:/data/hf_cache
    environment:
      - CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES:-0}
      - PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
      - OMP_NUM_THREADS=16
      - HF_HOME=/data/hf_cache
      - VLLM_CACHE_ROOT=/cache/vllm
      - TORCHINDUCTOR_CACHE_DIR=/cache/torchinductor
      - TRITON_CACHE_DIR=/cache/triton
    entrypoint: ["python3", "-m", "vllm.entrypoints.cli.main"]
    command:
      - serve
      - /models/Qwen3.8-27B-FP8-Pessoa
      - --served-model-name=${SERVED_MODEL_NAME:-qwen38-27b-pessoa}
      - --host=0.0.0.0
      - --port=${PORT:-8000}
      - --trust-remote-code
      - --quantization=compressed-tensors
      - --tensor-parallel-size=${TENSOR_PARALLEL_SIZE:-1}
      - --disable-custom-all-reduce
      - --kv-cache-dtype=${KV_CACHE_DTYPE:-fp8}
      - --block-size=128
      - --gpu-memory-utilization=${GPU_MEMORY_UTILIZATION:-0.96}
      - --max-model-len=${MAX_MODEL_LEN:-262144}
      - --max-num-seqs=${MAX_NUM_SEQS:-16}
      - --max-num-batched-tokens=${MAX_NUM_BATCHED_TOKENS:-16384}
      - --max-cudagraph-capture-size=64
      - --generation-config=vllm
      - --reasoning-parser=qwen3
      - --attention-backend=flashinfer
      - --enable-chunked-prefill
      - --enable-prefix-caching
      - --enable-auto-tool-choice
      - --tool-call-parser=qwen3_coder
      - --mm-encoder-tp-mode=data
      - --speculative-config={"method":"mtp","num_speculative_tokens":3}
      - >-
        --compilation-config={"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]

volumes:
  vllm-cache:
  hf-cache:

</details>

Configurable variables (set in .env or shell):

---

Provenance & Integrity

The quantized LM weights (shards 1–2) are byte-identical to the original text-only gptq-fp8-w8a16-mlp checkpoint. Shard 3 contains 333 BF16 visual + 15 BF16 MTP tensors extracted from the base model — no re-quantization was performed on these.

---

Hardware & Runtime

---

Reproduction

Step 1: FP8 W8A16 weight-only GPTQ quantization

# Inside Docker (vLLM infernal-invocation image, llm-compressor, SM120)
CUDA_VISIBLE_DEVICES=0 SUITE_DIR=/data/suite-v3 \
  python scripts/quant_gptq_unified.py \
  --model /data/models/Qwen3.8-27B-bf16 \
  --output /data/models/Qwen3.8-27B-gptq-fp8-w8a16-mlp \
  --format fp8-w8a16-mlp --calib-samples 181

Calibration tokens from $SUITE_DIR/tokens (default /data/suite-v3/tokens). Script: scripts/quant_gptq_unified.py. Format: fp8-w8a16-mlp (FP8 E4M3 per-channel weight-only for all Linear, BF16 activations, GPTQ Hessian correction).

Step 2: Merge visual tower + MTP from base

# Merge quantized LM shards with BF16 visual+mtp from base model
python /workspace/scripts/merge_vision_mixed.py \
  --src /models/Qwen3.8-27B-gptq-fp8-w8a16-mlp \
  --base /models/Qwen3.8-27B-bf16 \
  --output /data/models/Qwen3.8-27B-FP8-Pessoa

Script: scripts/merge_vision_mixed.py. Extracts 333 visual + 15 MTP tensors from the BF16 base, writes them as a 3rd shard, copies the 2 quantized shards unchanged, and patches config.json to Qwen3_5ForConditionalGeneration (VLM) with vision_config, text_config, and re:^mtp.* / re:.*visual.* added to the quantization ignore list.

---

Limitations

• Weight-only means slower prefill. BF16 activations require dequantization before GEMM, making prefill ~18% slower than W8A8 (4,875 vs 6,213 tok/s). Decode speed is comparable (45.6 vs 46.6 tok/s). If prefill throughput matters more than KLD, use the W8A8 variant (v17).
• Blackwell-optimized. FP8 E4M3 MMA is supported on SM120 (Blackwell), SM89/SM90 (Ada/Hopper), but the benchmark and KLD capture were run on SM120 only. Validate on your hardware.
• Body-only KLD. Fidelity measured against a shared BF16 LM head; end-to-end generation quality may differ slightly.
• Literary domain degradation. 5.1× higher KLD on literary text vs scientific — creative/prose generation is disproportionately affected, though less severely than in W8A8 or 4-bit configs.
• MTP speculative decoding acceptance depends on vLLM version and configuration. Ensure re:^mtp.* is in the quantization ignore list (it is, in this checkpoint) so vLLM's compressed-tensors loader leaves the BF16 MTP weights untouched.
• Quantization is lossy. Validate on your own workload before production use.

---

Acknowledgements

Standing on the shoulders of giants. This work would not exist without the rtx6kpro community and the broader local-inference-lab Discord — the open protocol, tooling, and baseline measurements that made systematic quantization comparison possible.

The distribution-fidelity (KLD) methodology follows the published protocol from Kimi-K3 distribution fidelity 1024×2048. The reference harness (glm52_exl3_shared_h_kld.py) and the benchmark tool (llm-inference-bench) are adapted from the rtx6kpro repository. The Gilded Gnosis EXL3 model cards established the per-bit error-ladder and validation-tier framework that this sweep builds on.

"I am the empty stage where the actors perform their various roles." — Bernardo Soares (heteronym of Fernando Pessoa), The Book of Disquiet