QwQ-Med-3-bf16

QwQ-Med-3 (BF16)

A BF16 conversion of yuvalkansal/QwQ-Med-3, a medical reasoning model in the QwQ-32B / Qwen2 family (Qwen2ForCausalLM, 32B parameters, 64 layers). The upstream weights are distributed as merged float32 (~131 GB); this repo is a shard-by-shard float32 → bfloat16 cast (~65.5 GB), numerically identical to running the F32 weights at BF16 and directly loadable by vLLM / Transformers without any further conversion.

This is the exact checkpoint used for the ICD-Bench evaluation below.

Evaluation

Scored on the 300-question ICD-Bench subset (from jha-lab/bottom-up-superintelligence), paper-faithful settings (chat template, Final Answer: X format, max_tokens=32768, temperature=0), vLLM BF16 weights with FP8 KV cache on a single H100 NVL:

Usage (vLLM)

vllm serve navindasg/QwQ-Med-3-bf16 \
  --dtype bfloat16 \
  --kv-cache-dtype fp8 \
  --max-model-len 34816

The chat template auto-prepends <think>\n; append the answer-format instruction to each question (\nPlease only output the choice letter in the answer field e.g. Final Answer: A) and parse Final Answer: X after the </think> tag.

Notes

• Hardware: 65.5 GB BF16 weights fit a single 80 GB GPU; use --kv-cache-dtype fp8 (or tensor-parallel across 2 GPUs) to leave room for a full 32K-token generation.
• License: inherits from the upstream yuvalkansal/QwQ-Med-3; verify terms there before redistribution or commercial use.
• A quantized W4A16 (int4 weights / bf16 activations) build for smaller GPUs is available at navindasg/QwQ-Med-3-W4A16.