Downloads · 30 days
35
2% of all-time downloads
MaliAir/KIMI-K3-MXFP4-MOE-Q8_0-GGUF
KIMI-K3-MXFP4-MOE-Q8_0-GGUF is a machine learning model from MaliAir. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repository contains the GGUF format conversion of moonshotai/Kimi-K3 , a powerful large language model developed by Moonshot AI. The conversion has been performed using the specialized llama.cpp fork that include…
Downloads · 30 days
35
2% of all-time downloads
All-time downloads
1.4K
Public
Repo size
1.6 TB
Likes
0
Public
Click a slice to open those files.
.gguf1.6 TB · 100%
From the Hugging Face model README
This repository contains the GGUF format conversion of moonshotai/Kimi-K3 , a powerful large language model developed by Moonshot AI. The conversion has been performed using the specialized llama.cpp fork that includes native support for the Kimi-K3 architecture.
Original Model Information Model Name: Kimi-K3
Developed by: Moonshot AI
Architecture: Custom architecture with specialized encoding_k3 module for efficient tokenization and processing
Context Length: [Insert the original model's context length]
Description: Kimi-K3 is a state-of-the-art large language model designed for complex reasoning, long-context understanding, and a wide variety of natural language processing tasks. For detailed information about capabilities, training data, and limitations, please refer to the original model card.
GGUF Conversion Details Prerequisites The conversion was performed using a specific environment with the following dependencies:
bash
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp git fetch origin pull/26185/head:pwilkin/kimi-k3-text git checkout pwilkin/kimi-k3-text Environment Setup bash
conda create -n kimi-convert python=3.10 conda activate kimi-convert
pip install huggingface-hub==0.17.3 pip install accelerate==0.24.1 pip install numpy==1.24.4 pip install transformers==4.31.0 Conversion Command The model was converted using the following command:
bash
export PYTHONPATH="/mnt/sdb/models--moonshotai--Kimi-K3/snapshots/9f62e4e9fffbd0a83ddd60e1c209d828994b3569:$PYTHONPATH" export NUMEXPR_MAX_THREADS=128
python -c "import encoding_k3; print('✓ encoding_k3 imported successfully!')"
python convert_hf_to_gguf.py /mnt/sdb/models--moonshotai--Kimi-K3/snapshots/9f62e4e9fffbd0a83ddd60e1c209d828994b3569
--outfile /mnt/sdb/KIMI-K3.GGUF
--outtype auto
Key Points about the Conversion:
The --outtype auto flag automatically selects the optimal output type (typically F16 for the base model)
The PYTHONPATH modification ensures the custom encoding_k3 module (Kimi-K3's specialized tokenizer) is accessible
The conversion leverages the specific PR branch that adds native Kimi-K3 support to llama.cpp
This KIMI-K3 base model powers our EASA-compliant High-Performance Computing (HPC) cluster, specifically architected for Aviation Safety and Compliance workflows at masi-ai.com.
KHM