Gemma-2-2B-it-Autoround-Safetensors

Gemma-2-2B-it Autoround

This is a post-training quantized version of google/gemma-2-2b-it, prepared as a Hugging Face checkpoint for LLiMa compilation and deployment on Sima.ai hardware.

Source revision: 299a8560bedf22ed1c72a8a11e7dce4a7f9f51f8

This checkpoint remains subject to the source model's license, intended-use guidance, and limitations.

Quantization

Calibration used the first 512 examples from HuggingFaceH4/ultrachat_200k train_sft, in deterministic order. Messages were rendered with the model chat template and packed into 512 complete 1,024-token spans. AutoRound batch size was 1. Calibration shuffling was disabled; the effective ordering is therefore deterministic.

Evaluation

Perplexity used the full wikitext-2-raw-v1 test split through the wikitext lm-eval task, without an example limit.

Raw JSON evidence is retained under perplexity_results/full_wikitext/.

Finite-scale validation and a Transformers chat-generation smoke test passed on 2026-07-19.

Reproduction

This directory includes the exact quantize.py, recipe.yaml, and versions.txt used for this artifact:

python quantize.py \
  --model-path /path/to/models--google--gemma-2-2b-it \
  --output-dir /path/to/Gemma-2-2B-it-Autoround-Safetensors

Environment

See versions.txt for the exact environment:

Python: 3.13.2
torch: 2.11.0+cu128
CUDA: 12.8
transformers: 5.10.1
llm-compressor: 0.12.0
auto-round: 0.13.0
compressed-tensors: 0.17.1

Deployment

This is the pre-LLiMa quantized Hugging Face artifact. Compile it with the appropriate LLiMa release and keep compiler output separate from this checkpoint. No upload is authorized as part of the current local batch.

Limitations

Quantization can affect quality, especially for tasks unlike the calibration data. Validate this checkpoint for the intended language, domain, prompt format, context length, safety requirements, and deployment runtime.