GLM-5.3-Flash-ABLITERATED-Colibri-INT4-gs64

GLM-5.3-Flash-ABLITERATED Colibrì INT4-gs64

Colibrì INT4-gs64 conversion of dealignai/GLM-5.3-Flash-ABLITERATED-FP8.

This conversion is designed for CPU inference with the Colibrì runtime.

Model Details

• Original developer: Z.ai
• Abliterated weights: dealignai
• Converted and shared by: disasters
• Architecture: GLM-5.3-Flash Mixture-of-Experts
• Total parameters: 321B
• Active parameters: Approximately 18B per token
• Source format: FP8
• Converted format: Colibrì INT4
• Group size: 64 (gs64)
• Model size: 194.7 GB
• Shards: 62
• Colibrì version: 1.10.2
• Runtime: CPU
• License: MIT

The routed experts were converted to grouped INT4. Quantization-sensitive components were retained at higher precision according to the Colibrì conversion policy.

Requirements

• Linux
• Fast local NVMe storage
• Approximately 210 GB of free disk space
• At least 25 GB RAM for streamed inference
• Approximately 200–230 GB RAM for full expert residency
• GCC with OpenMP support

The GLM-5.3-Flash engine in Colibrì 1.10.2 is CPU-only. A GPU is not required.

Download

python3 -m pip install --user -U "huggingface_hub[hf_xet]"
export PATH="$HOME/.local/bin:$PATH"
export HF_XET_HIGH_PERFORMANCE=1

hf download \
  disasters/GLM-5.3-Flash-ABLITERATED-Colibri-INT4-gs64 \
  --local-dir /path/to/glm53-flash-abliterated-colibri

Build Colibrì

git clone https://github.com/JustVugg/colibri.git
cd colibri/c

./setup.sh
make glm53 ARCH=native -j"$(nproc)"

The separate glm53 build target is required for GLM-5.3-Flash.

Verify the Model

./coli doctor \
  --deep \
  --model /path/to/glm53-flash-abliterated-colibri

The uploaded conversion passed coli doctor --deep:

• all 62 shards are present;
• safetensors headers are valid;
• required core tensors are present;
• tensor layouts are internally consistent;
• the RAM placement plan is viable;
• the quality-preserving conversion policy is enabled.

Interactive Chat

./coli chat \
  --auto-tier \
  --model /path/to/glm53-flash-abliterated-colibri

For systems with enough RAM to keep all experts resident:

PIN_GB=all DRAFT=0 \
./coli chat \
  --auto-tier \
  --model /path/to/glm53-flash-abliterated-colibri

Single Prompt

./coli run \
  --auto-tier \
  --model /path/to/glm53-flash-abliterated-colibri \
  --ngen 256 \
  "Explain how Mixture-of-Experts routing works."

Tested Performance

Test system:

• AMD EPYC 7763
• 64 CPU cores
• Approximately 250 GB RAM
• Full expert residency available
• CPU-only execution

Cold-run result:

64 tokens in 108.5 seconds
0.590 tokens/second
1.7 seconds/token

This is a single system-level measurement, not a comprehensive benchmark. Performance depends on CPU memory bandwidth, NUMA configuration, storage speed and cache state.

Conversion Details

The model was converted using Colibrì 1.10.2 with the following configuration:

Expert precision: INT4
I/O and sensitive component precision: 8-bit or higher
Group size: 64
Output size: 194.7 GB
Output shards: 62
Conversion time: approximately 3 hours

No fine-tuning or additional training was performed during conversion.

Compatibility

This checkpoint uses the Colibrì container format.

It is not directly compatible with:

• Transformers
• vLLM
• llama.cpp
• Ollama

Use the Colibrì runtime to load and run this model.

Limitations and Risks

• Quantization may change logits and reduce quality compared with the FP8 source.
• No independent MMLU, HarmBench or multilingual benchmark was performed on this converted checkpoint.
• The source model has undergone weight-level refusal modification.
• The model may generate inaccurate, biased or unsafe information.
• Do not rely on the model itself to enforce legal, safety or policy restrictions.
• Languages other than Russian, English and Chinese were not evaluated for this conversion.

Source Models and Credits

• Original model: zai-org/GLM-5.3-Flash
• Abliterated FP8 source: dealignai/GLM-5.3-Flash-ABLITERATED-FP8
• Runtime and converter: JustVugg/colibri

This repository contains only the Colibrì INT4-gs64 conversion. Credit for the original architecture and the abliterated source weights belongs to their respective creators.

Disclaimer

This model is provided for research and experimentation without warranty. Users are responsible for complying with applicable laws, licenses and platform policies.