Downloads · 30 days
74
5% of all-time downloads
jmoney54378256438905/GLM-5.3-Flash-4.65bpw-exl3
GLM-5.3-Flash-4.65bpw-exl3 is a image-text-to-text model from jmoney54378256438905. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for exllamav3. The card lists the license as mit.
Downloads · 30 days
74
5% of all-time downloads
All-time downloads
1.4K
Public
Parameters
94.7B
190 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors190 GB · 100%
How the weights are stored.
I1693.3B · 99%
From the Hugging Face model README
EXL3 quantization of GLM-5.3-Flash.
Quantized from zai-org/GLM-5.3-Flash-BF16
(the 642 GB BF16 master), not the default FP8 checkpoint — so the trellis is fitted to the
original weights rather than to already-rounded ones.
Requires ExLlamaV3 v1.4.5 or newer.
| Size on disk | 189.6 GB / 176.6 GiB, 35 shards |
| Effective bitrate | 4.67 bpw (requested 4.65 with -hq), head 6 bpw |
| Conversion | exllamav3 v1.4.5, -b 4.65 -hq, default 250×2048 calibration |
| MTP | included, inline as layers.45 — no separate mtp.safetensors |
| Vision tower | included |
Per-layer reconstruction over the 45 decoder layers: mean rfn 0.011789, mean sqnr 46.19 dB.
-hq allocates unevenly — layers 0-14 and 30-44 land at ~5.04-5.68 bpw while layers 15-29 sit at
~4.07-4.09, so read per-layer error against its own bitrate rather than a single threshold.
Needs roughly 210 GiB of VRAM for weights plus a large cache.
Measured on 4× CMP 170HX (64 GB each, sm_80) under TabbyAPI:
| metric | value |
|---|---|
| prefill | 924 tok/s @ 132,045-token prompt |
| decode | 29.7 tok/s |
| context | 262,144 (cache_size 1,048,576) |
| VRAM used | 214,796 MiB / 209.8 GiB |
| MTP acceptance | 61-74% |
Two notes if you serve it with TabbyAPI:
draft_mode: mtp finds it with no draft_model_name; the log should say
Using main model MTP component for drafting. A benign !! Replaced 6 tensors /
Overriding ...layers.45.* block at load is expected (dedup across the last two shards).gpu_split, not autosplit. Autosplit fills cards sequentially and packed
three of four to ~99%, leaving no room for the KDA prefill workspace: 625 tok/s prefill with
996 MiB free at peak. gpu_split_auto: false with gpu_split: [54, 54, 54, 54] gave
924 tok/s and 5.6 GB free — same footprint, 48% faster.Model by Z.ai. Quantization format and tooling by turboderp. Licensed MIT, as upstream.