gemma4-prometheus-exl3-6bpw

gemma4-prometheus-exl3-6bpw

<!-- polished-overview:start -->

Overview

gemma4-prometheus-exl3-6bpw is an EXL3-quantized checkpoint for ExLlamaV3-compatible runtimes, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

At a glance

What is included

• *.safetensors (4 files)
• config.json
• generation_config.json
• tokenizer.json
• tokenizer_config.json
• chat_template.jinja
• quantization_config.json
• Additional configuration, tokenizer, processor, or shard files (11 visible artifacts total)

Quick start

EXL3-compatible runtimes

Download the EXL3 files and load the desired bitrate with a current ExLlamaV3-compatible runtime. The correct loader and context settings depend on the model architecture and should be verified against the runtime's documentation.

Compatibility and responsible use

• Use a runtime that explicitly supports this format, architecture, and modality.
• Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
• Review the source model card and license before redistribution or deployment.
• Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
• Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.

<!-- polished-overview:end -->