BAR-2x7B-Tool-Use-GGUF

BAR-2x7B-Tool-Use — GGUF (first-of-its-kind FlexOlmo conversion)

This is the first GGUF conversion of allenai/BAR-2x7B-Tool-Use, one of AllenAI's BAR-family Mixture-of-Experts models released on 2026-04-19 based on the new FlexOlmo architecture.

⚠ Requires a FlexOlmo-enabled llama.cpp

Upstream llama.cpp does not support the FlexOlmo architecture: the support PR ggml-org/llama.cpp#22540 was closed unmerged on 2026-04-30. Use hob-llama.cpp, our llama.cpp build that adds FlexOlmo on a current upstream base:

• Prebuilt binaries: release hob-b11434: Linux (CPU x64/arm64, CUDA 12.8, ROCm, Vulkan), Windows (CPU x64/arm64, CUDA 12.4/13.4, ROCm, Vulkan) and macOS (Intel x64), with SHA256SUMS.txt.
• Or build from source:

git clone https://github.com/Seraphiel102/llama.cpp.git
cd llama.cpp
git checkout hob-b11434
cmake -B build -DGGML_CUDA=OFF        # or -DGGML_CUDA=ON etc. for your GPU
cmake --build build -j --target llama-cli llama-completion llama-server

What FlexOlmo is

Per transformers.models.flex_olmo, FlexOlmoDecoderLayer is Olmo2's hybrid post-norm decoder layer with the dense FFN swapped for OlmoE-style top-k MoE routing. Specifically:

• Attention with q_norm and k_norm (Olmo2-style)
• post_attention_layernorm and post_feedforward_layernorm (post-norm pattern, no input_layernorm)
• Top-k MoE FFN with softmax routing (OlmoE-style)
• No sliding-window attention

Model details

Quants

Ollama

Ollama's bundled llama.cpp does not support FlexOlmo, so this GGUF does not load in Ollama today. Use hob-llama.cpp (above). The Modelfile in this repo is kept for when Ollama adds support.

Usage — hob-llama.cpp

./build/bin/llama-completion \
    -m BAR-2x7B-Tool-Use.Q4_K_M.gguf \
    -p "Q: A train travels 60 miles in 1.5 hours. What's its average speed in mph?\nA:" \
    -n 60 --temp 0.3

Sample output:

"To find the average speed, we need to divide the distance by the time. So, the average speed is 60 miles / 1.5 hours = 40 mph."

Validation

Q4_K_M was tested on three prompts during development:

• Factual: "The capital of France is" → "Paris."
• Reasoning: 60mi/1.5h speed problem → "40 mph" with steps
• Code/explanation: fibonacci docstring → coherent explanation of memoization

All clean stops on the model's EOS, no looping or degeneration.

Re-verified 2026-10-06 on hob-llama.cpp b11434: the prompt "The capital of France is" gives byte-identical output and identical top-5 logprobs to the April build (temp 0, 32 tokens).

License

Apache 2.0 (matching the source release at AllenAI). This conversion is a derivative work — same license applies.

Conversion details

• Source: allenai/BAR-2x7B-Tool-Use (downloaded 2026-04-29)
• Tools: our FlexOlmo llama.cpp patch (April 2026); now carried as patch 5 of hob-llama.cpp
• Steps: convert_hf_to_gguf.py → llama-quantize
• Upstream PR ggml-org/llama.cpp#22540 closed unmerged; support lives in hob-llama.cpp

Acknowledgments

• AllenAI for the BAR family release and the FlexOlmo architecture
• llama.cpp maintainers — the existing Olmo2 + OlmoE handlers gave us the right primitives to combine
• This conversion produced for the Zenith swarm project — autonomous engineering collective

Citation

If you use this GGUF, please cite AllenAI's BAR release:

@misc{allenai2026bar,
  title  = {BAR: Beam-and-Adjust-Routing models},
  author = {{Allen Institute for AI}},
  year   = {2026},
  month  = {April},
  url    = {https://huggingface.co/allenai/BAR-2x7B-Tool-Use}
}