Apertus-v1.5-8B-FP8

Apertus v1.5 8B -- FP8

Quantized checkpoint of Apertus v1.5 8B.

Full-precision FP8 quantization of swiss-ai/Apertus-v1.5-8B using the FP8_DYNAMIC scheme on all linear projections (excluding lm_head, embeddings, and vision/audio tokenizers which remain in BF16).

Use this checkpoint when maximum quality is critical and GPU memory allows.

Thanks to: Swiss AI, blancsw, Anunay-Yadav, Oleg, Cyrilvallez, AryanAhadinia, robmsmt, and Neural Magic. See Attribution for details.

Usage (vLLM)

Until Apertus 1.5 support is available in a native vLLM release, use the ghcr.io/swiss-ai/vllm_apertus_1.5_release image. The build recipe and patched Dockerfile are available at github.com/swiss-ai/model-launch/tree/main/images/vllm_apertus_1.5.

# Download the checkpoint
huggingface-cli download onprem-ai/Apertus-v1.5-8B-FP8 \
  --local-dir ./apertus-v1.5-8b-fp8

# Serve it
docker run --gpus all -v ./apertus-v1.5-8b-fp8:/model \
  ghcr.io/swiss-ai/vllm_apertus_1.5_release:latest-amd64 \
  --model /model \
  --served-model-name apertus-v1.5-8b \
  --host 0.0.0.0 --port 8080 \
  --dtype auto \
  --chat-template /model/chat_template.jinja \
  --tool-call-parser apertus --enable-auto-tool-choice \
  --reasoning-parser apertus

Quantization Process

Single-pass calibration-free quantization using llm-compressor 0.12.0:

scheme: FP8_DYNAMIC
targets: Linear
ignore: lm_head, model.language_model.embed_tokens, model.embed_tokens,
        re:.*vision_tokenizer.*, re:.*audio_tokenizer.*

Limitations

• vLLM only. SGLang does not support compressed-tensors checkpoints.
• Requires vLLM with Apertus1p5 arch support. Use the ghcr.io/swiss-ai/vllm_apertus_1.5_release image or build from upstream PR #50496.

Attribution

• Original model: swiss-ai/Apertus-v1.5-8B by Swiss AI, Apache 2.0
• GPU-native multimodal refactor: blancsw at Infomaniak
• Upstream PR: vllm-project/vllm #50496 by Anunay-Yadav
• Reasoning parser: AryanAhadinia
• Quantization tooling: llm-compressor by Neural Magic