gemma4-prometheus-workflow

Gemma4 Prometheus workflow

<!-- polished-overview:start -->

Overview

gemma4-prometheus-workflow is a development repository associated with the model-building workflow, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

At a glance

What is included

• checkpoints/--home--op--models--gemma4--google_gemma-4-31B-it.jsonl
• eval_results.json
• export_prometheus_merged.py
• gemma4_prometheus.toml
• quantize_gemma4_prometheus.py
• scripts/eval_gemma4.py

Quick start

Getting started

Start with the upstream library named in the repository metadata and keep all configuration, tokenizer, processor, and weight files together. This repository is an artifact release, so the source project remains the authoritative reference for task-specific loading code.

Compatibility and responsible use

• Use a runtime that explicitly supports this format, architecture, and modality.
• Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
• Review the source model card and license before redistribution or deployment.
• Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
• Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.

<!-- polished-overview:end -->

Reproduction scripts and config for the Gemma4 Prometheus run that led to the exported merged model and GPTQ output.

What is included

• gemma4_prometheus.toml
• export_prometheus_merged.py
• quantize_gemma4_prometheus.py
• checkpoints/ with the Prometheus journal file

Related repos

• Fixes used to make the pipeline work: groxaxo/gemma4-prometheus-fixes
• Merged model: groxaxo/gemma4-prometheus-merged
• GPTQ model: groxaxo/gemma4-prometheus-gptq-4bit
• Upstream source model: google/gemma-4-31B-it

Run

conda activate gemma4-prometheus-ready

export CUDA_VISIBLE_DEVICES=GPU-828df6fd-3fd0-ed25-0b2b-2b6d9d8dca47,GPU-78996a05-18c5-e153-b621-096273299d41,GPU-89c6bfdc-6f42-d312-de77-a9fb1ae370d8
export PM_CONFIG=./gemma4_prometheus.toml

prometheus --config ./gemma4_prometheus.toml --non-interactive --overwrite-checkpoint
python export_prometheus_merged.py --config ./gemma4_prometheus.toml --output-dir ./merged-model
python quantize_gemma4_prometheus.py --config ./gemma4_prometheus.toml --model-dir ./merged-model --output-dir ./gptq-4bit --offload-dir ./gptq-offload --prompts-per-dataset 16

Notes

• The workflow assumes the local Prometheus and GPTQModel fixes described in the fixes repo.
• The GPTQ step uses gptqmodel==5.8.0 plus the Gemma4-specific patch set.
• The checkpoint journal is included so the exported merged model can be traced back to the exact Prometheus trial.