Downloads · 30 days
179
7% of all-time downloads
Kimi-K2-Instruct-DRAFT-0.6B-v3.0-GGUF
jukofyork/Kimi-K2-Instruct-DRAFT-0.6B-v3.0-GGUF
Kimi-K2-Instruct-DRAFT-0.6B-v3.0-GGUF is a machine learning model from jukofyork. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A 0.6B parameter draft (speculative decoding) model for use with Kimi-K2-Instruct.
All-time downloads
2.5K
Public
Repo size
2.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf1.3 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Aug 10, 2025
- Updated
- Aug 11, 2025
- SHA
- 7b91a367
Base models
Model card
From the Hugging Face model README
A 0.6B parameter draft (speculative decoding) model for use with Kimi-K2-Instruct.
See Kimi-K2-Instruct-DRAFT-0.6B-v3.0 for the models in transformers format, and a detailed explanation of how the model was created.
I've included the Q4_0 quants for 3 different context lengths:
- Kimi-K2-Instruct-DRAFT-0.6B-32k-Q4_0.gguf
- Kimi-K2-Instruct-DRAFT-0.6B-64k-Q4_0.gguf
- Kimi-K2-Instruct-DRAFT-0.6B-128k-Q4_0.gguf
NOTES:
- The 14 heads of
Qwen2.5-0.5Bdoesn't allow for any of the other 4-bit quants to be made (and experimentation has shown using more or less than 4-bits for speculative decoding is a waste of time anwyay). - Due to
llama.cppusing "static-YaRN" the scaling factor remains constant regardless of input length! Only use the longer context versions when processing long contexts is required... - If you want to recreate these, then the
TikToken/SentencePiecetokenizer mismatch requires a small hack toconvert_hf_to_gguf.py(see main model page for details).
- License
- apache-2.0
- Created
- Aug 10, 2025
- Updated
- Aug 11, 2025