ggk

ggk

One package for working with GGUF models locally: an OpenAI-compatible LLM server, a diffusion image/video/audio generator and a GGUF metadata/tensor editor with a built-in quantizer — three panels on one GUI, powered by one unified engine compiled in a single build on top of gk, an independent tensor library. There is no ggml anywhere in the tree.

Install

pip install ggk

The build compiles the bundled engine (CPU by default, Metal on macOS). GPU backends are opt-in at install time:

GGK_CUDA=1 pip install ggk     # NVIDIA
GGK_HIP=1 pip install ggk      # AMD ROCm
GGK_VULKAN=1 pip install ggk   # Vulkan

Each switch drives the whole engine — the server, the diffusion runtime and the multimodal projectors all evaluate their graphs on the one gk build.

Run

ggk                 # unified GUI — Server / Diffuser / Editor panels
python -m ggk       # same thing

Each panel also runs on its own, exactly like the standalone gguf-server / gguf-diffusion / gguf-editor packages did:

ggk server          # LLM server GUI
ggk diffuser        # image generation GUI
ggk editor          # GGUF editor GUI

And the engines are directly scriptable from the CLI:

ggk server engine -- --model model.gguf
ggk diffuser engine -- -m sd.gguf -p "a lighthouse at dusk" -o out.png
ggk editor quantize -m in.gguf -o out-q4_k.gguf --type q4_k
ggk editor devices

For examples, test the diffusion model in this repo:

ggk diffuser engine -- -m test-nvfp4.gguf -p "fox" -o fox.png
ggk diffuser engine -- -m test-nvfp4.gguf -p "cow" -o cow.png
ggk diffuser engine -- -m test-nvfp4.gguf -p "dog" -o dog.png
ggk editor test-nvfp4.gguf

add --diffusion-fa tag (turn on flash attention for diffusion model) to diffuser engine will get significantly faster process

<Gallery />

*gk is our own experimental kernel, recently support multiGPU tensor split, new features are coming very soon, please see reference for details

Layout

vendor/engine/           the unified ggk engine (one CMake build)
  gk/                    the gk compute kernels (CPU + optional GPU backends)
  gk/compat/             the historical ggml C API, implemented on gk
  src/ common/ mtmd/     GGUF LLM runtime
  app/                   the gguf-server HTTP server
  diffusion/             diffusion runtime + CLI
  quantizer/             quantizer shared library (its own quant kernels)
src/ggk/                 the Python package
  server/ diffuser/ editor/   the three panels (backend + web frontend each)
  gui.py static/         the unified 3-panel GUI shell

Nothing above gk/compat/ knows gk exists: the runtimes include the same ggml.h / ggml-backend.h / gguf.h headers and call the same functions they always did, while graph building, allocation, scheduling and the kernels themselves are gk's. See vendor/engine/README.md for the engine's own build options.

The editor's quantizer stays independent — its qz_* codec is compiled both into the quantizer library the editor drives and into gk itself, so the encoder and the runtimes' decoder can never disagree about a GGUF block.

Documentation

• about ggk
  • panel server
    • supported models
  • panel diffuser
    • supported models
  • panel editor
    • internal quantizer

more examples for multimedia generation

image (get test files: model + vae + clip + adapter)

ggk diffuser engine -- --diffusion-model pixart-nvfp4.gguf --vae pig_pixart_vae_fp16-f16.gguf --llm pig_clip-nvfp4.gguf --llm-adapter pig_t5_adapter-f16.gguf -p "close-up portrait of dog" --diffusion-fa -v -o out.png

video (get test files: model + vae + clip + adapter)

ggk diffuser engine -- -M vid_gen --diffusion-model wan2.1_t2v_1.3b-q4_0.gguf --vae pig_wan_vae_fp32-f16.gguf --llm pig_clip-nvfp4.gguf --llm-adapter pig_umt5_adapter-f16.gguf -p "a pig moving quickly in a beautiful winter scenery nature trees sunset tracking camera" --cfg-scale 6.0 --sampling-method euler -v -n "blurry ugly bad" -W 480 -H 480 --diffusion-fa --offload-to-cpu --video-frames 14 -o out.avi

audio (get test files: model + vae + encoder)

ggk diffuser engine -- --diffusion-model ace-step-v1-3.5b-q4_k_m.gguf --vae pig_ace_vae_fp32-f16.gguf --llm umt5base-q4_0.gguf -p "pop, upbeat, female vocals" --lyrics "[verse]\nMorning light filtering through the pine\n[Chorus]\nSoftly the world begins to breathe" --audio-duration 30 --steps 40 --offload-to-cpu --diffusion-fa -v -o out.wav

<audio controls src="https://huggingface.co/gguf-org/ggk/resolve/main/morning.wav"></audio>

audio checkpoint (get test file here)

ggk diffuser engine -- -m yue2-3b-nvfp4.gguf -p "pop, upbeat, female vocals" --lyrics "[Verse]\nMorning light filtering through the pine\n[Chorus]\nSoftly the world begins to breathe" --audio-duration 30 --steps 8 --diffusion-fa -v -o out.wav

<audio controls src="https://huggingface.co/gguf-org/yue2-3b-gguf/resolve/main/out.wav"></audio>

edit (get test files: model + vae + clip + adapter + mmproj) and sheep picture (or any picture)

ggk diffuser engine -- --diffusion-model mageflow-edit-turbo-nvfp4.gguf --vae pig_mageflow_vae_fp32-f16.gguf --llm pig_clip-nvfp4.gguf --llm-adapter pig_qwen3vl_4b_adapter-f16.gguf --llm_vision mmproj-qwen3vl-4b-it-f16.gguf --ref-image sheep.png -p "a sheep in sunglasses" --cfg-scale 1.00 --steps 4 --sampling-method euler --diffusion-fa -v -o out.png

pig-clip compatible

support natively trained pig clip for low vram devices; check it out here

Reference

pig engine - the new gguf compute kernels (gk)