antares-1b-mlx-bf16

antares-1b-mlx-bf16

Unquantized bfloat16 MLX build of fdtn-ai/antares-1b, Cisco Foundation AI's file-level vulnerability-localization model.

No quantization, no re-training, no weight surgery: every retained tensor is bit-for-bit the original checkpoint. This repo exists only because the upstream checkpoint does not load in mlx-lm as published.

Why this repo exists

mlx_lm.load("fdtn-ai/antares-1b") fails twice:

1. Missing rope_theta. The upstream config.json (exported with Transformers 5.x) carries the RoPE base only inside the nested rope_parameters block. mlx_lm's granitemoehybrid.ModelArgs requires a top-level rope_theta, which is a required field with no default — so construction fails. Here the top-level key is present with the correct value (10000000). This matters for quality, not just loading: silently falling back to a default base would wreck long-context attention.

2. Duplicate lm_head.weight.

   ValueError: Received 1 parameters not in model: lm_head.weight.
   

   The config sets tie_word_embeddings: true, so the MLX model has no separate lm_head module, but the checkpoint still ships the tensor. Transformers ties it to the embedding at load time; mlx-lm loads weights strictly and rejects the extra key.

   Before dropping it, the tensor was compared against model.embed_tokens.weight over all 100,352 rows as raw uint16 — bitwise identical, confirming the weights are genuinely tied and nothing is lost. Transformers agrees: it reports lm_head.weight.data_ptr() == model.embed_tokens.weight.data_ptr().

   Dropping the duplicate also shrinks the checkpoint from 3.7 GB to 3.3 GB.

Everything else — tokenizer.json, tokenizer_config.json, generation_config.json, chat_template.jinja — is copied unchanged from upstream.

Verification

Logits were compared against the Transformers reference implementation (GraniteMoeHybridForCausalLM, transformers==5.14.1, CPU) on a 1,020-token prompt, across all 1,020 positions and the full 100,352-token vocabulary:

Read the last two rows together: this build's deviation from the float32 reference is no larger than what Transformers' own bfloat16 path produces. At matched precision (float32 row) the two implementations agree to within floating-point accumulation order. There is no systematic error.

Usage

pip install mlx-lm

mlx_lm.generate --model dkavocado/antares-1b-mlx-bf16 --prompt "def add(a, b):" --ignore-chat-template

Roughly 80 tok/s on an M5 Pro, ~3.3 GB peak memory.

Serving for Antares CLI

Antares CLI talks to a streaming OpenAI-compatible POST /v1/completions endpoint and builds the Granite prompt itself, so it needs the raw completion text including the <tool_call>…</tool_call> blocks.

mlx_lm.server infers a json_tools tool-call parser from chat_template.jinja and strips those blocks out of the text field. Antares then sees an empty answer and every scan ends as Model ended without an explicit final submission with zero tool calls. Override the template with a neutral one to disable that parser:

mlx_lm.server --model dkavocado/antares-1b-mlx-bf16 \
  --host 127.0.0.1 --port 8001 \
  --decode-concurrency 8 \
  --chat-template "{% for m in messages %}{{ m['content'] }}{% endfor %}"

Antares profile (~/.antares/profiles.toml):

[profiles.mlx]
model = "dkavocado/antares-1b-mlx-bf16"
backend = "remote"
endpoint = "http://127.0.0.1:8001/v1/completions"
context_window = 32768

[profiles.mlx.generation]
max_tokens = 8192
temperature = 0.3
top_p = 1.0
frequency_penalty = 0.3
stop_tokens = ["<|end_of_text|>", "<|start_of_role|>"]
use_completions_api = true

mlx_lm.server resolves the request's model field as a path or repo id, so the profile's model value must be the same string the server was started with (or default_model).

Model behavior and limits

Unchanged from upstream: the model localizes vulnerabilities at file level — it returns ranked candidate file paths for a given CWE. It does not give line numbers, code snippets, or fixes, and results vary between identical runs. Treat every result as a lead to verify.

License and attribution

Apache 2.0, inherited from the original model. The weights are the work of Cisco Foundation AI — see fdtn-ai/antares-1b for the authoritative model card, evaluation results, and intended-use guidance. The upstream repository is gated behind an acceptance click; please read its terms there.

This repository redistributes those weights in MLX layout for Apple Silicon users, with no modification beyond the two loader fixes described above.