hal0-brain-sft-fpx8-agent

hal0-brain ROCmFP8 Agent

ROCmFPX ROCmFP8 Agent quantization of Hal0ai/hal0-brain-sft for hal0's brain role.

[!WARNING] This is not a stock Q8_0 GGUF. It contains the custom Q8_0_ROCMFPX tensor type (GGML type ID 103) and requires a ROCmFPX-capable runner. Stock llama.cpp, Ollama, LM Studio, and other upstream-only GGUF runners must reject this file rather than attempt inference.

Artifact contract

ROCmFPX-aware inspection reports 219 tensors:

• 58 × Q8_0_ROCMFPX (custom type ID 103)
• 112 × Q8_0 (type ID 8)
• 49 × F32 (type ID 0)

The mixed tensor policy is expected for the Agent preset: it keeps selected agent-coherence-sensitive tensors on protected types while using ROCmFPX FP8 weights elsewhere. general.file_type is 115 (Q8_0_ROCMFPX_AGENT).

Required runner

Use only a runner explicitly declaring rocmfpx-gguf compatibility. The hal0 catalog allows this model only with its rocmfpx and vulkanfpx runners.

Known compatible hal0 runner:

ghcr.io/hal0ai/hal0-rocmfpx@sha256:fd6b02a720e633e402e929e19eedefff52aeec18e5de8f43e525689e523985f3

That image provides ROCm/HIP and Vulkan backends and includes MiniCPM5 tokenizer support needed by this model. Do not substitute a stock llama.cpp image.

Typical direct invocation inside a compatible image is:

llama-server \
  --model /models/model.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -ngl 999 \
  -fa on

Select ROCm0 or Vulkan0 according to the host and runner configuration.

Provenance

The artifact was generated from the immutable F16 GGUF:

Repository: Hal0ai/hal0-brain-sft-GGUF
Revision:   6b190df6e816cc806f7fa7ae3de7248f5551e00b
File:       model.gguf
SHA-256:    ed9d28c4eac1d7c291bc80d9410c243a3d28e655921ccaf90f2b6619aa24d2c3

Quantization used charlie12345/ROCmFPX commit 61f2f2d7bc4955e9bca821095ef69125837133b5 and its agent wrapper:

SRC=/path/to/model-F16.gguf \
OUT=/path/to/model.gguf \
FORMAT=rocmfp8 \
PROFILE=agent \
scripts/quantize-rocmfpx-agent.sh

The resolved preset was Q8_0_ROCMFPX_AGENT.

Verification

The published digest was calculated over the complete file. Tensor types were read with the ROCmFPX fork's gguf-py reader, not inferred from repository or filename labels.

A current stock llama.cpp build was also tested as a negative compatibility check and rejected the file at the first custom tensor:

tensor 'blk.3.attn_k.weight' has invalid ggml type 103

This rejection is expected and required.