Bielik-11B-v3.0-Instruct-NVFP4

Bielik-11B-v3.0-Instruct-NVFP4

NVFP4 (4-bit floating-point) weight-only quantization of speakleash/Bielik-11B-v3.0-Instruct, built for fast inference on NVIDIA Blackwell GPUs and designed to pair with DFlash speculative decoding for the fastest Bielik serving setup.

Community quantization. Unofficial build, not an official SpeakLeash release.

What this is

Hardware

NVFP4 runs natively on NVIDIA Blackwell (GB10 / DGX Spark, B200, RTX 50-series). Built and tested on NVIDIA GB10 (Grace Blackwell).

Usage (vLLM)

Fastest: NVFP4 + DFlash speculative decoding

vllm serve norecyc/Bielik-11B-v3.0-Instruct-NVFP4 \
  --served-model-name bielik-11b \
  --speculative-config '{"method":"dflash","model":"speakleash/Bielik-11B-v3.0-DFlash","num_speculative_tokens":8}' \
  --max-model-len 32768 --gpu-memory-utilization 0.85 \
  --host 0.0.0.0 --port 8000

Plain (no speculative decoding)

vllm serve norecyc/Bielik-11B-v3.0-Instruct-NVFP4 --max-model-len 32768

Chat template

Use the model's native chat template (bundled chat_template.jinja, ChatML-style). Do not use a tool-calling template (e.g. Hermes) for plain chat — it makes the model emit tool calls instead of answering.

Evaluation

Quality vs the BF16 base, measured with lm-evaluation-harness (loglikelihood / multiple-choice — speculative decoding does not affect these scores, so this isolates quantization quality), n=200 per task:

Both differences are within statistical noise (~±3% at n=200) — no measurable quality regression from 4-bit quantization. (n=200 rules out degradation beyond ~3-4%; for sub-1% certainty, run full task sets.)

Performance

Single-stream decode throughput on one NVIDIA GB10 (temp=0), decomposing the two speedup levers — NVFP4 quantization x DFlash speculative decoding:

• NVFP4 alone 3.5x, DFlash alone 2.7x, combined 7.6x over the BF16 / no-speculation baseline.
• Sub-multiplicative (2.7 x 3.5 = 9.5x) because the draft stays BF16 and becomes the relatively larger per-step cost once the target is 4-bit.
• Optimal num_speculative_tokens = 8; speculative mean acceptance length ~3.2 of 8.

Quantizing the DFlash draft to NVFP4 is not currently loadable in vLLM (the dedicated qwen3_dflash draft loader does not support compressed-tensors weight packing), so the draft is served in BF16.

Quantization recipe

from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

recipe = QuantizationModifier(targets="Linear", scheme="NVFP4A16", ignore=["lm_head"])
oneshot(model="speakleash/Bielik-11B-v3.0-Instruct", recipe=recipe, output_dir="bielik-nvfp4")

Attribution & license

• Base: SpeakLeash — speakleash/Bielik-11B-v3.0-Instruct (Apache-2.0).
• Speculative draft: speakleash/Bielik-11B-v3.0-DFlash (Apache-2.0).
• Released under Apache-2.0, preserving the base model's license and attribution.