hrm-text-agent-gguf

HRM-Text Code + Agent GGUF

This directory bundles two specialized HRM-Text checkpoints. Each model is provided as a canonical F16 GGUF and a Q8_0 derivative. The complete model-specific Jinja template is embedded in every GGUF as tokenizer.chat_template.

Contents

The source Safetensors for both checkpoints are F32, so F16 is the canonical 16-bit conversion. Q8_0 is quantized from that F16 GGUF. Each file contains 259 tensors: 258 model tensors in F16 or Q8_0 plus the F32 recurrent latent tensor.

The local package uses hard links for the four large files, so it does not consume another 7.25 GB in this workspace. Copying or uploading this directory carries the files normally.

Build llama.cpp

Stock llama.cpp does not support hrm_text. Apply the included patch to the exact validated commit:

git clone https://githubfast.com/ggml-org/llama.cpp.git
cd llama.cpp
git checkout 6a257d44633d4a752183ed778b88d2924d0a6b9d
git apply ../runtime/llama.cpp-hrm_text.patch
cmake -B build -DGGML_METAL=ON -DGGML_NATIVE=OFF -DLLAMA_BUILD_UI=OFF
cmake --build build --config Release --target llama-server llama-quantize -j

Use -DGGML_METAL=OFF on systems without Metal. CUDA builds may use the normal llama.cpp CUDA options.

Code example

From this directory, start the Q8_0 code model:

./llama.cpp/build/bin/llama-server \
  --model models/HRM-Text-1B-Code-Q8_0.gguf \
  --alias hrm-code --host 127.0.0.1 --port 8080 \
  --ctx-size 512 --batch-size 512 --ubatch-size 512 --parallel 1 \
  --n-gpu-layers all --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --jinja --cache-ram 0 --no-cache-idle-slots --no-webui

In another terminal:

python3 examples/code.py
python3 examples/code.py "Write a Python function is_even(n). Return only the function."

The Python example uses llama-server's OpenAI-compatible chat endpoint. The embedded template supplies the code model's required synth,cot condition.

Tool example

Stop the code server, then start Agent v2. The --special flag is required by the minimal raw tool loop so <tool_call> tokens remain visible:

./llama.cpp/build/bin/llama-server \
  --model models/HRM-Text-1B-Agent-v2-Q8_0.gguf \
  --alias hrm-agent --host 127.0.0.1 --port 8081 \
  --ctx-size 1024 --batch-size 1024 --ubatch-size 1024 --parallel 1 \
  --n-gpu-layers all --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --jinja --special --cache-ram 0 --no-cache-idle-slots --no-webui

In another terminal:

python3 examples/tool.py

Expected shape:

calculator('23 * 47') -> 1081
The product of 23 and 47 is 1081.

The example calls /apply-template, /tokenize, and /completion, parses the model's JSON tool request, executes a restricted local calculator, feeds the result back through the embedded template, and prints the final answer. It uses only the Python standard library.

Runtime constraints

• HRM-Text is a PrefixLM. Keep the complete initial prompt in one physical batch, so --batch-size and --ubatch-size must be at least the prompt token count.
• Keep cache_prompt disabled. Do not reuse a shorter prompt's KV cache or use speculative multi-token decoding with this patch.
• F16 and Q8_0 were validated with 129/129 layers on Metal, Flash Attention, Q8_0 K/V cache, and non-causal PrefixLM prefill.
• The code model and tool model are separate experts. Route tasks between them; the author found that merging their weights destroys one skill or the other.

Validation summary

• Agent v2 Q8_0 emitted valid calculator and parallel weather JSON calls, consumed tool results, returned correct final answers, and avoided a tool call for an irrelevant request.
• Code Q8_0 matched the F32 checkpoint byte-for-byte at the generated-text level on two greedy synth,cot probes. One simple function was correct; the mergesort probe reproduced the F32 model's undefined merge() defect.
• The embedded Jinja text was read back from every GGUF and matched its source template exactly. See SHA256SUMS for artifact verification.

About the older Code-Tools-SFT checkpoint

The older pzarzycki/hrm-text-1b-code-tools-sft checkpoint should not be compared to these models by row count alone:

• Its published Stage A saw 38,248 selected rows once, totaling 10,000,147 response tokens and 4,781 optimizer updates. It was a partial pilot over a 1.13M-row code corpus, not 38k tool-call examples.
• Sampled Stage A records contain code responses but no <tools>, <tool_call>, or <tool_result> markers. Tool trajectories appear in Stage B, which the publisher explicitly says was not trained or released.
• Jason's code model used about 25k unique examples for three epochs, or roughly 75k example exposures, in the correct synth,cot lane. Agent v2 used a dedicated function-calling mixture and reports BFCL results.

Our earlier tool test did use PrefixLM, direct, and the checkpoint's native Jinja, but it asked Stage A for a skill it had not learned. That failure does not establish that the checkpoint is a bad tool model; it establishes that the released checkpoint is not actually a trained tool model.

The code picture is mixed. A detailed larger(a, b) request produced correct code with both the publisher's raw direct envelope and the native Jinja. A vague "Write mergesort in Python" request was echoed, while a longer factorial request produced syntactically valid but semantically broken code. The fair label is "unevaluated Stage A code pilot with unstable quality", not "30k-sample tool model".

Sources and license

• jasoncarreira/hrm-text-code, revision 969a37192d76b7a989dc1e055f38f33e15ff86c9
• jasoncarreira/hrm-text-agent-v2, revision f6b8eb6d82f0dbe2455b80930daa84979a411b8f
• Patched llama.cpp commit 6a257d44633d4a752183ed778b88d2924d0a6b9d

Both Jason checkpoints are marked CC-BY-NC-4.0 and should be treated as research / non-commercial artifacts. Review the upstream model cards and dataset licenses before redistribution or use.