Reasoning-Medical-27B-NVFP4

Reasoning-Medical-27B-NVFP4

Blackwell-oriented ModelOpt quantization of EpistemeAI/Reasoning-Medical-27B, pinned to revision e5cfbcc2498df44c07124f670a75c77179fc8e67. The exported safetensors total 21.92 GB.

Quantization

• NVIDIA Model Optimizer: 0.45.0
• Recipe: huggingface/qwen3_5/ptq/w4a16_nvfp4-fp8_attn-kv_fp8_cast
• MLP projections and lm_head: W4A16 NVFP4
• Self-attention and the large linear-attention projections: FP8
• KV cache metadata: FP8 cast
• Preserved at source precision: vision tower, MTP, convolution and sensitive Gated-DeltaNet projections selected by the NVIDIA recipe
• Calibration: 512 CNN/DailyMail text samples of at most 512 tokens; medical benchmark cases were not used
• Source license: Apache-2.0

Full provenance and SHA-256 hashes are in quantization_manifest.json.

vLLM

vllm serve switzerchees/Reasoning-Medical-27B-NVFP4 \
  --served-model-name EpistemeAI/Reasoning-Medical-27B \
  --quantization modelopt \
  --kv-cache-dtype bfloat16 \
  --max-model-len 32768 \
  --reasoning-parser qwen3 \
  --limit-mm-per-prompt '{"image": 1, "video": 0}'

BF16 KV cache is recommended for accuracy and is forced explicitly above. The exported recipe contains FP8 KV-cache metadata; with vLLM 0.23, leaving the flag at auto therefore selects FP8 rather than the model's BF16 dtype. FP8 KV serving should be treated as a separate accuracy/performance trade-off and validated for the target runtime.

Limitations and safety

This quantized derivative has not been clinically validated. The upstream model card reports medical benchmarks but no independent clinical validation and no substantive medical-image benchmark. Quantization can introduce additional errors. Do not use this model as a substitute for a physician, for autonomous diagnosis, or for treatment decisions. German-language quality is evaluated separately and is not implied by the upstream model's English language tag.