Downloads · 30 days
159
3% of all-time downloads
jukofyork/GLM-4.5-DRAFT-0.6B-v3.0-GGUF
GLM-4.5-DRAFT-0.6B-v3.0-GGUF is a machine learning model from jukofyork. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A 0.6B parameter draft (speculative decoding) model for use with GLM-4.5, GLM-4.5-Air and GLM-4-32B-0414.
Downloads · 30 days
159
3% of all-time downloads
All-time downloads
6.2K
Public
Repo size
1.3 GB
Likes
19
Public
Click a slice to open those files.
.gguf1.3 GB · 100%
From the Hugging Face model README
A 0.6B parameter draft (speculative decoding) model for use with GLM-4.5, GLM-4.5-Air and GLM-4-32B-0414.
See GLM-4.5-DRAFT-0.6B-v3.0 for the models in transformers format, and a detailed explanation of how the model was created.
I've included the Q4_0 quants for 3 different context lengths:
Qwen2.5-0.5B doesn't allow for any of the other 4-bit quants to be made (and experimentation has shown using more or less than 4-bits for speculative decoding is a waste of time anwyay).llama.cpp using "static-YaRN" the scaling factor remains constant regardless of input length! Only use the longer context versions when processing long contexts is required...