Ornith-1.0-35B-ROCmFPX-GGUF

Ornith-1.0-35B — ROCmFPX quants for AMD Strix Halo

ROCmFPX-family GGUF quantization of deepreinforce-ai/Ornith-1.0-35B (Qwen3.5-MoE architecture, multimodal), produced for AMD Strix Halo (Ryzen AI Max, gfx1151) and the hal0 home inference platform.

⚠️ These files require the Hal0ai/Hal0_ROCmFPX llama.cpp fork (or the ghcr.io/hal0ai/hal0-rocmfpx container image that hal0 uses). Stock llama.cpp will reject the tensor types (invalid ggml type 101).

See also the companion repo: gsrunion/Ornith-1.0-9B-ROCmFPX-GGUF.

Files

Measured performance

On AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB unified LPDDR5X, ROCm backend, hal0-rocmfpx image, 64K ctx):

• Q4_0_ROCMFP4_STRIX_LEAN: ~66 tok/s decode — the MoE's small active-parameter set decodes faster than the dense 9B (42 tok/s) despite the 3.7× file size.

How it was made

BF16 GGUF source and imatrix from unsloth/Ornith-1.0-35B-GGUF, quantized with the Hal0_ROCmFPX fork's llama-quantize:

llama-quantize --imatrix imatrix_unsloth.gguf_file \
  Ornith-1.0-35B-BF16.gguf Ornith-1.0-35B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN

Serving

Directly with the fork's llama-server (as done on hal0 boxes):

llama-server -m Ornith-1.0-35B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --mmproj mmproj-BF16.gguf -ngl 999 -fa on --jinja -c 65536

Or on hal0: download into the model store, register via add-from-path, then hal0 slot create <name> --type llm --hardware rocm --model <id> and load via a long-lived curl.

Credits

• Base model: deepreinforce-ai — Ornith-1.0-35B (MIT)
• BF16 GGUF conversion + imatrix: unsloth
• ROCmFPX quant formats + fork: Hal0ai