qwen3.5-9b-anwar-hermes-GGUF

<p align="center"> <img src="assets/qwen35-anwar-hermes-cover.jpg" alt="Qwen3.5-9B-Anwar-Hermes model overview by AnwarPy" width="720"> </p>

Qwen3.5-9B-Anwar-Hermes — GGUF Q4_K_M

A compact GGUF release derived from Qwen/Qwen3.5-9B, fine-tuned for tool selection, structured output, function calling, and multi-step agent workflows in English and Arabic.

Built for Hermes Agent

Qwen3.5-9B-Anwar-Hermes is the result of a month-long training, dataset-curation, refinement, and evaluation cycle focused on practical compatibility with Hermes Agent and its tool ecosystem.

The model was shaped around function calling, structured tool schemas, terminal workflows, multi-step chains, and agent-style instructions. The goal is to provide a local model that works naturally with Hermes workflows—not a generic chat model with tool use added as an afterthought. Compatibility was validated against the documented Hermes-oriented behavior and terminal gates; it is not a claim of perfect parity across every tool, runtime, or future Hermes release.

At a glance

Performance snapshot

The benchmark covered 69 sequential scenarios with 52 tool definitions at temperature 0.0 and seed 42. These are task-specific results, not a claim of broad production readiness. The complete reproducibility summary is available in evals/tool-eval-bench-summary.md.

Quick start

1. Download

hf download AnwarPy/qwen3.5-9b-anwar-hermes-GGUF \
  qwen3.5-9b-anwar-hermes-Q4_K_M.gguf \
  --local-dir .

2. Start llama.cpp

llama-server \
  -m qwen3.5-9b-anwar-hermes-Q4_K_M.gguf \
  --alias qwen3.5-9b-anwar-hermes \
  -ngl 99 -c 8192 -np 1 \
  --host 127.0.0.1 --port 11442 \
  --jinja --reasoning off --reasoning-budget 0

3. Verify the API

curl http://127.0.0.1:11442/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3.5-9b-anwar-hermes",
    "messages": [{"role": "user", "content": "Reply with exactly: MODEL_OK"}],
    "temperature": 0
  }'

Expected assistant content:

MODEL_OK

The exact command above was validated against the published GGUF with llama.cpp commit e8bcb772d749a581d717b79ac8b9cd57301d9651. The same artifact passed load and inference canaries with official Windows llama.cpp b10078 (f04801018) and LM Studio engine 2.25.2. Runtime flags may vary in other versions.

Recommended use

• Function calling and tool selection.
• Strict or structured JSON generation.
• Multi-step local agent workflows.
• English and Arabic assistant tasks.
• Local OpenAI-compatible inference through llama.cpp-based runtimes.

This release is text/tool-use oriented. Vision capability inherited from the base model was not qualified for this GGUF release and should not be assumed.

Files and verification

Release GGUF SHA-256:

cf0cc3541a75739861c4208b47a7f163c8273f0e56b4132b9bebf2a357a2c7c3

License

Apache License 2.0. See LICENSE. Users are also responsible for reviewing the base-model and dataset terms.

Acknowledgements

Built from Qwen3.5 by the Qwen team and trained with pinned data from Nous Research, the Hermes Agent reasoning-trace dataset, and the audited derived terminal curriculum sourced from Roman1111111/gpt5.5-terminal. Quantization and inference use the GGUF and llama.cpp ecosystem.