antares-1b-bf16-GGUF

antares-1b-bf16-GGUF

Unquantized bfloat16 GGUF build of fdtn-ai/antares-1b, Cisco Foundation AI's file-level vulnerability-localization model.

Every published GGUF of this model so far is quantized (Q2_K … Q8_0). This one is not: weights are the original bf16 values, converted only in container format.

How it was built

python convert_hf_to_gguf.py fdtn-ai/antares-1b --outtype bf16 --outfile antares-1b-bf16.gguf

llama.cpp build 10090 (720d7fa), converter class GraniteHybridModel.

One caveat if you reproduce this: the upstream config.json (exported with Transformers 5.x) keeps the RoPE base only inside the nested rope_parameters block. Add a top-level "rope_theta": 10000000 before converting and then verify it landed in the GGUF — a silent fall back to the default base would quietly wreck long-context attention:

python -c "from gguf import GGUFReader; r=GGUFReader('antares-1b-bf16.gguf'); print(r.fields['granite.rope.freq_base'].contents())"
# 10000000.0

This build's metadata was checked: rope.freq_base 10000000, attention.scale 0.0078125, embedding_scale 12.0, residual_scale 0.22, logit_scale 8.0, block_count 40, head_count 16, head_count_kv 4, vocab_size 100352.

Measurements

Perplexity over 9 chunks of Python source at n_ctx 2048, and llama-bench on an Apple M5 Pro (Metal), same machine, same llama.cpp build:

So the honest trade-off: BF16 gives you the exact checkpoint and ~1.4 % lower perplexity, Q8_0 gives you 1.65× faster generation at half the memory. If you are running batched agentic workloads where throughput dominates, Q8_0 is the pragmatic choice; if you are validating results or care about matching the reference implementation, use this one.

Usage

llama-server -m antares-1b-bf16.gguf --alias antares-1b --host 127.0.0.1 --port 8003 \
  -np 8 -c 65536 --metrics

Note that -c is the total context split across -np slots — the command above gives each of 8 slots 8192 tokens.

With Antares CLI

Antares needs a streaming OpenAI-compatible POST /v1/completions endpoint and builds the Granite prompt itself, so it needs the raw completion text including <tool_call>…</tool_call> blocks. llama.cpp does not touch those on the completions route, so no workaround is needed here (unlike mlx_lm.server, which strips them via its json_tools parser).

[profiles.llama]
model = "antares-1b"
backend = "remote"
endpoint = "http://127.0.0.1:8003/v1/completions"
context_window = 32768

[profiles.llama.generation]
max_tokens = 8192
temperature = 0.3
top_p = 1.0
frequency_penalty = 0.3
stop_tokens = ["<|end_of_text|>", "<|start_of_role|>"]
use_completions_api = true

Keep context_window at or below the per-slot context, otherwise long investigations get rejected by the server.

Related

• MLX build of the same unquantized weights, for Apple Silicon: DKAvocadoo/antares-1b-mlx-bf16

Model behavior and limits

Unchanged from upstream: the model localizes vulnerabilities at file level — it returns ranked candidate file paths for a given CWE, without line numbers, snippets, or fixes, and results vary between identical runs. Treat every result as a lead to verify, not as proof.

License and attribution

Apache 2.0, inherited from the original model. The weights are the work of Cisco Foundation AI; see fdtn-ai/antares-1b for the authoritative model card, evaluations, and intended-use guidance. The upstream repository is gated behind an acceptance click — please read its terms there. This repository only changes the container format.