Mach-1-Additive-35B-Multimodal-GGUF

Mach-1-Additive-35B — GGUF

GGUF build of Mach-1-Additive-35B for local inference on consumer hardware (NVIDIA CUDA, Windows/Linux Vulkan, CPU; on Apple Silicon the codec runs on CPU — use Mach Studio for GPU inference on macOS).

[!IMPORTANT] This GGUF requires the Mach-1 fork of llama.cpp: SyzygyResearch/llama.cpp-mach1. Mainline llama.cpp will not load it — the model uses custom trellis-coded tensor payloads and decode ops that only exist in the fork.

Files

Quick start

Build the fork — pick the backend for your hardware (NVIDIA → CUDA, AMD/Intel → Vulkan):

git clone https://github.com/SyzygyResearch/llama.cpp-mach1
cd llama.cpp-mach1

# NVIDIA (requires the CUDA toolkit)
cmake -B build -DGGML_CUDA=ON
# AMD / Intel / other (requires the Vulkan SDK, incl. glslc)
cmake -B build -DGGML_VULKAN=ON

cmake --build build --config Release -j

Run directly from Hugging Face:

./build/bin/llama-cli -hf SyzygyResearch/Mach-1-Additive-35B-GGUF

or from a local download:

# interactive chat
./build/bin/llama-cli -m Mach-1-Additive-35B.mach1.gguf

# single-turn / scripted use (without -st, llama-cli enters interactive
# conversation mode after answering the prompt)
./build/bin/llama-cli -m Mach-1-Additive-35B.mach1.gguf -st -p "your prompt"

<!-- Theme-aware embed. If your renderer ignores <picture>, use the plain image line underneath instead and delete this block. -->

<!-- Plain fallback:  -->

Benchmarks

Per-benchmark Retention (Score divided by that of the respective full precision model)

Mach-1 Small's own scores and teacher scores:

Speed