Gemma-4-31B-it-Gewell_BF16

Gemma 4 31B IT — Gewell BF16

This is the full-BF16 Gemma 4 31B IT text bundle for Gewell, a single-GPU inference engine specialized for NVIDIA Blackwell (sm_120/sm_120a). This repository also includes the extracted BF16 vision tower and multimodal projector. The text bundle uses Gewell's native schema-6 format; it is not a Transformers, vLLM, or llama.cpp checkpoint.

Bundle contents

• weights.gwt — 61,395,791,872 bytes (57.18 GiB), full-BF16 text weights.
• manifest.json — artifact identity, tensor inventory, layout, source provenance, and checksums.
• tokenizer.json — tokenizer required by the Gewell runtime.
• vision.safetensors — 1,151,536,616 bytes (1.07 GiB), BF16 vision tower and multimodal projector.

The weight file SHA-256 is 3ff1f3ae4310383cac803789a78bfa6e205b16f404d8e4b3ef9ed00bd817ca5e; its payload SHA-256 is dc5925d7e2403c2ab72101db6233f66c0edd3a46b8023af2fa25f82bbe437d32. The vision file SHA-256 is 01121d84f3261586f7da72166293183f5b44b0c704ef334ea13de4f263e1d6a9. The manifest also records the header, entry-table, serving-asset, and per-tensor checksums.

Precision

All 832 physical text tensors are BF16, with one logical alias for the tied output head. No text weights are quantized. The schema-6 artifact currently uses Gewell's mixed-v2 container identifier, but its manifest inventory is entirely BF16.

Download and verify

Follow the Gewell build instructions, then download this repository and the pinned official MTP assistant. The assistant is loaded directly in its original Google safetensors format and is not redistributed here.

Use a current hf CLI for this bundle. weights.gwt exceeds 50 GB, so a basic browser or HTTP download is not supported by the Hub's standard file endpoint.

MODEL_DIR=/absolute/path/to/Gemma-4-31B-it-Gewell_BF16
ASSISTANT_DIR=/absolute/path/to/gemma-4-31B-it-assistant
ASSISTANT_REV=4735700dca7bd22fad5dc348c228b50ec6cbac6d

hf download LeDissolution/Gemma-4-31B-it-Gewell_BF16 \
  --local-dir "$MODEL_DIR"

hf download google/gemma-4-31B-it-assistant model.safetensors \
  --revision "$ASSISTANT_REV" \
  --local-dir "$ASSISTANT_DIR"

build/gewell verify "$MODEL_DIR/weights.gwt"

Use hf auth login first if your Hugging Face setup requires authentication for the upstream Google repository.

The pinned assistant file is 939,042,560 bytes and has SHA-256 9f80df6099fa1fd7db71220ec9ee864d5ecff769878697dd5763e4285b15a1da.

Launch with vision and MTP

This example targets a 96 GiB RTX PRO 6000 Blackwell and enables image input plus three assistant proposals per speculative cycle:

build/gewell \
  --vision "$MODEL_DIR/vision.safetensors" \
  --assistant "$ASSISTANT_DIR/model.safetensors" \
  --mtp-depth 3 \
  serve-http \
  --model-dir "$MODEL_DIR" \
  --max-batch 1 \
  --kv-cache-gpu-mib 8192 \
  --model gemma-4-31b-bf16

Omit --vision to disable image input. Set --mtp-depth 0 to disable MTP; the assistant is loaded only for a positive depth. Vision and assistant weights add approximately 1.15 GB and 0.94 GB of GPU allocation respectively, before vision scratch and KV cache. The BF16 bundle is not intended for a 32 GiB GPU; adjust the cache budget and batch limit for your hardware and workload.

The engine was developed and tested on an RTX PRO 6000 Blackwell. See the launch guide and HTTP API documentation for current hardware requirements, image request format, and serving options.

Scope and provenance

This repository contains the BF16 text model and its compatible BF16 vision component. The speculative-decoding assistant remains a separate optional component downloaded from the pinned official Google revision above.

The text and vision components were derived from google/gemma-4-31B-it at revision fcf2302760ae9c6e528a8dbba9dd636e56848237. The text converter preserves the source BF16 values without quantization. Evaluate model behavior on your own workloads.

Use is subject to the upstream Gemma 4 license.