hal0-brain-sft-fpx4-agent

hal0-brain-sft-fpx4-agent (GGUF ROCmFP4)

ROCmFP4-quantized GGUF of Hal0ai/hal0-brain-sft for the hal0 agent slot on AMD Strix Halo / RDNA accelerators.

The quant type is Q4_0_ROCMFP4_STRIX_LEAN (the charlie12345/rocmfp4-llama fork's variant of llama.cpp's quantize). Standard gguf-py reports the tensor type as 101 (undefined in the stock enum) — this file requires the ROCm FP4 fork of llama-server, not upstream.

How hal0 uses this file

The hal0 agent slot is the default LLM anchor (ADR-0023). On AMD ROCm boxes (Strix Halo, RDNA3+), the recommended assign is this GGUF, loaded into a [email protected] container using the hal0-rocmfpx runtime image:

hal0 model pull Hal0ai/hal0-brain-sft-fpx4-agent-GGUF
hal0 model assign hal0-brain-sft-fpx4-agent --slot agent
hal0 slot load agent

The container's hal0-rocmfpx image (ghcr.io/hal0ai/hal0-rocmfpx) bundles a llama-server built with the ROCmFP4 patch and /dev/kfd + /dev/dri passed through, so the slot picks up FP4 matmul automatically. Suggested flags (from the r105 brain-agent-fpx slot profile):

--jinja -fa on -ngl 99 -dev ROCm0 -b 512 -ub 512 \
--threads 16 --threads-batch 32 --no-mmap --metrics --no-webui

Required runtime

This file uses the AMD ROCm FP4 tensor type (raw_dtype=101) which is not recognized by upstream llama.cpp. You need one of:

• charlie12345/rocmfp4-llama — checkout mtp-rocmfp4-strix and build:

  git clone https://github.com/charlie12345/rocmfp4-llama.git
  cd rocmfp4-llama && git checkout mtp-rocmfp4-strix
  env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh
  

• A downstream llama.cpp / llama-server that has merged the ROCm FP4 patch.
• hal0's hal0-quant-fc toolbox image, which bundles a compatible binary.
• The ghcr.io/hal0ai/hal0-rocmfpx hal0-slot image (recommended for the hal0 stack).

Usage — hal0-slot (the official wiring, ROCm container)

# 1. Pull the GGUF into hal0's model store
hal0 model pull Hal0ai/hal0-brain-sft-fpx4-agent-GGUF

# 2. Assign to the agent slot
hal0 model assign hal0-brain-sft-fpx4-agent --slot agent

# 3. Load — container runtime image is selected from slot config
#    (hal0.toml [slots.agent] profile="chat" device="gpu-vulkan" → hal0-rocmfpx image on ROCm boxes)
hal0 slot load agent

# 4. Verify
curl http://127.0.0.1:8081/v1/models

Usage — direct llama-cli (from the ROCmFP4 fork build)

./build-strix-rocmfp4-mtp/bin/llama-cli -m model.gguf \
    -p "What does the hal0 'agent' slot default to?" -n 256 --jinja

Usage — direct llama-server (production ROCm)

./build-strix-rocmfp4-mtp/bin/llama-server \
    -m model.gguf \
    --jinja -fa on -ngl 99 -dev ROCm0 \
    -b 512 -ub 512 --threads 16 --threads-batch 32 \
    --no-mmap --metrics --no-webui \
    --port 8081

Tradeoffs vs. base

Model details

Related

• Hal0ai/hal0-brain-sft — BF16 safetensors (base)
• Hal0ai/hal0-brain-sft-GGUF — F16 GGUF (upstream llama.cpp)
• Hal0ai/hal0-brain-sft-fpx8-agent — Q8_0 GGUF (agent slot, CPU/VRAM-tight)
• Collection: hal0-brain

License

Apache-2.0.

Provenance

• Repo: Hal0ai/hal0-brain-sft-fpx4-agent
• Org: Hal0ai
• Project: https://github.com/Hal0ai/hal0
• Build: charlie12345/rocmfp4-llama @ mtp-rocmfp4-strix