Downloads · 30 days
13
3% of all-time downloads
RepublicOfKorokke/gemma-4-E4B-it-oQ4-fp16
gemma-4-E4B-it-oQ4-fp16 is a machine learning model from RepublicOfKorokke. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
This model was quantized using oQ (oMLX v0.3.6) mixed-precision quantization. float16 gives ~20% faster prefill on M1/M2 Apple Silicon (native fp16). bfloat16 is safer on M3/M4 and for numerical stability.
Downloads · 30 days
13
3% of all-time downloads
All-time downloads
509
Public
Parameters
7.7B
4.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors4.8 GB · 99%
How the weights are stored.
U327.5B · 98%
From the Hugging Face model README
This model was quantized using oQ (oMLX v0.3.6) mixed-precision quantization. float16 gives ~20% faster prefill on M1/M2 Apple Silicon (native fp16). bfloat16 is safer on M3/M4 and for numerical stability.
| Model | File size | MMLU | JMMLU | HELLASWAG | ARC_CHALLENGE | GSM8K |
|---|---|---|---|---|---|---|
| gemma-4-e4b-it-4bit | 4.89 GB | 64.3% | 60.3% | 73.7% | 84.0% | 85.3% |
| gemma-4-E4B-it-oQ3-fp16 | 3.59 GB | 18.3% | 34.3% | 38.0% | 52.3% | 3.0% |
| gemma-4-E4B-it-oQ4-fp16 | 4.46 GB | 61.7% | 61.3% | 74.3% | 82.7% | 88.3% |
| Model | Benchmark | Accuracy | Correct | Total | Time(s) |
|---|---|---|---|---|---|
| gemma-4-e4b-it-4bit | MMLU | 64.3% | 193 | 300 | 418.2 |
| gemma-4-E4B-it-oQ3-fp16 | MMLU | 18.3% | 55 | 300 | 544.7 |
| gemma-4-E4B-it-oQ4-fp16 | MMLU | 61.7% | 185 | 300 | 352.2 |
| gemma-4-e4b-it-4bit | JMMLU | 60.3% | 181 | 300 | 192.3 |
| gemma-4-E4B-it-oQ3-fp16 | JMMLU | 34.3% | 103 | 300 | 295.4 |
| gemma-4-E4B-it-oQ4-fp16 | JMMLU | 61.3% | 184 | 300 | 161.7 |
| gemma-4-e4b-it-4bit | HELLASWAG | 73.7% | 221 | 300 | 219.3 |
| gemma-4-E4B-it-oQ3-fp16 | HELLASWAG | 38.0% | 114 | 300 | 203 |
| gemma-4-E4B-it-oQ4-fp16 | HELLASWAG | 74.3% | 223 | 300 | 177.2 |
| gemma-4-e4b-it-4bit | ARC_CHALLENGE | 84.0% | 252 | 300 | 176.5 |
| gemma-4-E4B-it-oQ3-fp16 | ARC_CHALLENGE | 52.3% | 157 | 300 | 149 |
| gemma-4-E4B-it-oQ4-fp16 | ARC_CHALLENGE | 82.7% | 248 | 300 | 150.3 |
| gemma-4-e4b-it-4bit | GSM8K | 85.3% | 256 | 300 | 1486.1 |
| gemma-4-E4B-it-oQ3-fp16 | GSM8K | 3.0% | 9 | 300 | 1441 |
| gemma-4-E4B-it-oQ4-fp16 | GSM8K | 88.3% | 265 | 300 | 1110.9 |
| Model | Test Case | TTFT (ms) | TPOT (ms) | pp TPS | tg TPS | E2E (s) | Throughput | Peak Mem |
|---|---|---|---|---|---|---|---|---|
| gemma-4-e4b-it-4bit | pp4096/tg128 | 4898.2 | 16.99 | 836.2 tok/s | 59.3 tok/s | 7.056 | 598.7 tok/s | 4.71 GB |
| gemma-4-e4b-it-4bit | pp32768/tg128 | 49735.7 | 32.28 | 658.8 tok/s | 31.2 tok/s | 53.835 | 611.1 tok/s | 5.49 GB |
| gemma-4-E4B-it-oQ4-fp16 | pp4096/tg128 | 3192.1 | 15.21 | 1283.2 tok/s | 66.3 tok/s | 5.124 | 824.4 tok/s | 4.83 GB |
| gemma-4-E4B-it-oQ4-fp16 | pp32768/tg128 | 36248.8 | 26.43 | 904.0 tok/s | 38.1 tok/s | 39.606 | 830.6 tok/s | 5.63 GB |