Downloads · 30 days
2
4% of all-time downloads
alegsandyr/gemma_3_12B_qkv512_ff1620_q4_g32_full_submission
gemma_3_12B_qkv512_ff1620_q4_g32_full_submission is a machine learning model from alegsandyr. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This submission runs the Czech-to-German compressed Gemma 3 12B system.
Downloads · 30 days
2
4% of all-time downloads
All-time downloads
56
Public
Parameters
1B
5.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.3 GB · 99%
How the weights are stored.
U82.6B · 69%
From the Hugging Face model README
This submission runs the Czech-to-German compressed Gemma 3 12B system.
Default Hugging Face repository for this self-contained submission:
alegsandyr/gemma_3_12B_qkv512_ff1620_q4_g32_full_submission
The best locally validated quality-first q4 checkpoint from the latest experiments is:
outputs/models/gemma_3_12B_qkv512_ff1620_q4_g32_gptq_calib1024_infprompt
It was quantized from:
alegsandyr/gemma_3_12B_qkv512_ff1620
This checkpoint uses all 1024 calibration examples with calibration_field=inference_prompt.
The Hugging Face repository is intended to contain these weights at the repository root together
with this submission runner.
The checkpoint uses a custom SVD-factorized format with factor quantization. The submission bundle includes the minimal loader needed to instantiate SVDLinear = up(down(x)), wrap quantized factors when quantization_config.json is present, and run Hugging Face generation.
bash setup.sh
The evaluator may also preinstall dependencies from requirements.txt.
Official entrypoint:
bash run.sh \
--lang-pair ces-deu \
--batch-size 8 \
--input input.txt \
--output output.txt
The script writes exactly one translation line for each input line. Logs and progress messages go to stderr.
The official --batch-size argument is accepted. To reduce OOM risk with long WMT inputs, the runner internally caps the effective micro-batch size to MAX_EFFECTIVE_BATCH_SIZE (default 1). Set MAX_EFFECTIVE_BATCH_SIZE=8 if the evaluation GPU has enough memory and full batching is desired.
By default, run.sh first checks whether model weights are present next to run.sh.
In a clone/snapshot of the self-contained Hugging Face repository, it loads the local repository
root directly. If weights are not present next to the script, it falls back to:
alegsandyr/gemma_3_12B_qkv512_ff1620_q4_g32_full_submission
For local smoke tests before upload, set:
MODEL_DIR=outputs/models/gemma_3_12B_qkv512_ff1620_q4_g32_gptq_calib1024_infprompt bash run.sh ...
Smaller local experiments were also tested:
outputs/models/gemma_3_12B_qkv512_ff1620_q4_g64_gptq_calib1024_infprompt is about 4.7G, but smoke tests showed occasional English fragments.outputs/models/gemma_3_12B_qkv512_ff1620_q4_g48_gptq_calib1024_infprompt is about 4.8G, but smoke tests showed word artifacts.outputs/models/gemma_3_12B_qkv512_ff1620_q4_g128_gptq_calib128 is about 4.5G, but quality dropped more noticeably.The model repository is expected to contain:
config.json
tokenizer files
model.safetensors or model.safetensors.index.json with shards
compression_config.json
quantization_config.json
ces-deu / cs-de.bf16.max_new_tokens=256.DEQUANTIZE_SVD_ON_LOAD=1) to avoid repeated per-forward dequantization overhead.DEVICE_MAP=auto to use Accelerate dispatch.