shieldgemma-9b-awq-4bit

shieldgemma-9b-awq-4bit

Reproducible AWQ 4-bit quantization of google/shieldgemma-9b at revision b8b636016df4540721a098c7aab91c97ec6ee508.

Reproducibility

• Pipeline version: 2.0.0
• Quantizer: GPTQModel==7.1.0
• Source dtype: bfloat16
• Calibration data: 500 frozen C4 samples (250 English + 250 Portuguese)
• Calibration SHA-256: 65ea2fdb18470289314defc0fb94e27d4877b62143569615a1ce01745607ce1f
• C4 revision: 1588ec454efa1a09f29cd18ddd04fe05fc8653a2
• Random seed: 42

Quantization configuration

{
  "group_size": 128,
  "sym": false,
  "desc_act": false,
  "true_sequential": true,
  "lm_head": false,
  "pack_dtype": "int32",
  "format": "gemm",
  "scale_search_chunked_activations": true,
  "bits": 4
}

Calibration call

{
  "batch_size": 4,
  "calibration_sort": "desc",
  "calibration_concat_size": null,
  "calibration_data_min_length": 10,
  "calibration_concat_separator": null
}

The complete runtime environment, resolved checkpoint metadata, and file hashes are stored in runtime_environment.json, quantization_manifest.json, and SHA256SUMS.

vLLM runtime compatibility

This checkpoint uses asymmetric AWQ (sym=false, with an explicit zero-point) because ShieldGemma's Gemma 2 architecture is served in BF16. The zero-point configuration enables vLLM's BF16-compatible Marlin path; the symmetric AWQ configuration falls back to the regular FP16-only AWQ kernel and is therefore incompatible with this model's required inference dtype. It was validated with vLLM 0.31.0 using --dtype bfloat16 and automatic kernel selection.

Versioning

The checkpoint that preceded this reproducible build is preserved under the v1-original tag. Consumers that require an immutable dependency should pin a tag or commit SHA instead of relying on main.