halogen-qwen3.8-27b

<p align="center"> <img src="halogen.jpg" alt="halogen — the peon's inference engine" width="760"> </p>

halogen — Qwen3.8-27B for AMD Strix Halo

Weights for halogen, a dedicated inference engine for Qwen3.8-27B on AMD Strix Halo (gfx1151).

These weights are in halogen's own .hgn format and will not load in transformers, vLLM, or llama.cpp. They exist to be mounted into the halogen container.

What is here

qwen3.8-27b-p1w4d-d2.hgn      35.9 GB   checkpoint
tokenizer/                              flat tokenizer directory

Use

hf download peonist-ai/halogen-qwen3.8-27b --local-dir ~/halogen-models

docker run --rm -p 8731:8731 \
  --device /dev/kfd --device /dev/dri --group-add keep-groups \
  --security-opt seccomp=unconfined --ipc=host \
  -v ~/halogen-models:/models:ro \
  -v ~/halogen-models/tokenizer:/tokenizer:ro \
  ghcr.io/peonist-ai/halogen:0.1.4

An OpenAI-compatible endpoint comes up on :8731. Full documentation is in the engine repository.

Precision

~6.3 bits/weight effective at decode. Two thirds of the bytes decode streams are 8-bit or wider — the aggressive 4-bit path is fenced to prefill and never touches token generation.

The only 4-bit trunk tensors carry values calibrated by someone else — the NVFP4 quantization published in unsloth/Qwen3.8-27B-NVFP4 — rather than values we produced. Homemade quantization is used only where no donor exists.

Speculative decoding weights

This checkpoint additionally carries two drafter heads, both trained by us:

• an MTP head fine-tune, and
• DFlash2, a block drafter (5 layers, ~2.2B parameters).

Speculative decoding with either is byte-identical to serial greedy decode — the drafter only proposes, and a token is emitted only if the full model would have produced it. It is a speed optimization with no quality cost, and it can be disabled per request.

Provenance and license

Derived from Qwen3.8-27B, with 4-bit values imported from unsloth/Qwen3.8-27B-NVFP4. These weights are a derivative work and are governed by the license of the original model — see the base model for those terms. Verify them yourself before commercial use; nothing here grants rights the upstream license does not.

The halogen engine is separate, closed source, and distributed under its own terms — see the engine repository.