gemma4-e2b-exp-quant

sub1quant mixed-budget Gemma 4 E2B artifacts

This repository contains the mixed-budget sub-4-bit artifact from sub1quant. The base model is not mirrored here; download google/gemma-4-E2B separately.

Current artifact

The checkpoint contains 316 language-model weight tensors:

Live Colab evaluation

Run date: 2026-06-29

Hardware/runtime: NVIDIA L4, CUDA, dense BF16 evaluation after applying the quantized weights.

This supports a narrow claim: BF16-baseline-equivalent perplexity on this exact Gemma4/WikiText/Colab runner at about 4.00 BPW. It is not an FP16 result, not an FP8 comparison, and not a throughput result. The current evaluator reconstructs/applies weights into a normal dense model for correctness.

Result files:

• eval_results/mixed_budget_full_g128_target4p0_ppl_live.json
• eval_results/base_full_ppl_live.json
• eval_results/mixed_budget_live_colab_comparison.json
• eval_results/mixed_budget_scan_full_g128_target4p0.json

Reproduce

pip install "transformers>=5.5.0" torch accelerate safetensors huggingface_hub

python -c "from huggingface_hub import snapshot_download; snapshot_download('google/gemma-4-E2B', local_dir='./models/gemma-4-E2B')"

python scripts/limited_ppl_bench.py \
  --label mixed_budget_full_g128_target4p0 \
  --model-dir models/gemma-4-E2B \
  --wikitext data/wiki.test.txt \
  --quantized-pt quantized/gemma_mixed_budget_full_g128_target4p0.pt \
  --tokens 1000000000 \
  --max-length 512 \
  --stride 512 \
  --device cuda \
  --output eval_results/mixed_budget_full_g128_target4p0_ppl_live.json

License

The quantization code and metadata in this repository are Apache-2.0. The base model remains governed by Google's Gemma license.