antares-1b-mlx-8bit

antares-1b-mlx-8bit

8-bit MLX build of fdtn-ai/antares-1b, Cisco Foundation AI's file-level vulnerability-localization model, quantized from the bfloat16 MLX conversion at DKAvocadoo/antares-1b-mlx-bf16.

About 1.74 GB on disk. This is the build to use if you want Antares in LM Studio.

Why this repo exists

No LM Runtime found for model format 'torchSafetensors'!

That is the error LM Studio shows when you point it at an unquantized MLX conversion, and it is why the excellent bf16 build upstream cannot be used as is.

LM Studio decides whether a folder is an MLX model or a PyTorch checkpoint by looking at config.json. An unquantized MLX conversion writes a config that is indistinguishable from a plain Transformers checkpoint, so LM Studio classifies it as PyTorch, finds no runtime that can serve it, and stops. mlx_lm.convert -q adds a quantization block, and that block is what makes the folder recognisable as MLX.

Comparing the two configs as parsed JSON rather than line by line:

keys: 8bit=51  bf16=49

only in the 8-bit config:
   quantization         {"group_size": 64, "bits": 8, "mode": "affine"}
   quantization_config  {"group_size": 64, "bits": 8, "mode": "affine"}

only in the bf16 config:        (none)
keys differing in value:        (none)

Everything else that appears changed in a line diff comes from mlx_lm re-indenting and re-ordering keys. Note that quantizing here is not a performance improvement, it is just what makes the model loadable in LM Studio.

The upstream bf16 repo's two loader fixes — the top-level rope_theta that mlx-lm requires, and the dropped duplicate lm_head.weight that the config declares tied — are inherited intact. You can confirm rope_theta: 10000000 is still present at the top level of this repo's config.json.

How it was made (so you can do it yourself)

pip install mlx-lm
hf download DKAvocadoo/antares-1b-mlx-bf16 --local-dir ~/models/antares-1b-mlx-bf16

mlx_lm.convert \
    --hf-path ~/models/antares-1b-mlx-bf16 \
    --mlx-path ./antares-1b-mlx-8bit \
    -q --q-bits 8 --q-group-size 64

No re-training and no weight surgery. Because every retained tensor in the bf16 source is bit-for-bit the original checkpoint, quantizing from it is equivalent to quantizing from the gated original, with no compounding loss.

The group size of 64 is deliberate rather than the default. Granite 4.0 is a hybrid Mamba architecture, and state-space layers are more sensitive to coarse quantization than plain attention layers, so the finer grouping is cheap insurance.

Using it in LM Studio

Either download through LM Studio's model search, or sideload:

hf download JLinNZ/antares-1b-mlx-8bit \
    --local-dir ~/.lmstudio/models/JLinNZ/antares-1b-mlx-8bit

LM Studio needs the two-level publisher/model-name directory layout and picks the model up on next launch. Turn the local server on in the Developer tab and it serves an OpenAI-compatible endpoint on http://localhost:1234.

Antares is an agent, not a chat model. It expects a CWE description and a terminal, and it replies with <tool_call> blocks. Chatting with it directly will look broken.

One useful difference from mlx_lm.server: the upstream bf16 card documents that mlx_lm.server infers a json_tools parser from the chat template and strips <tool_call> blocks out of the response text, so an agent loop sees an empty answer and every scan ends with no tool calls. LM Studio does not do this — the raw <tool_call> text arrives intact, which is one less thing to work around.

Verification

This build was checked end to end. The numbers below come from eleven files across six languages, nine carrying deliberate weaknesses, two written correctly as controls, with a published answer key held outside the scanned directory so the model cannot read it. Measured with a local harness built for this model and a purpose-built sample tree.

This is a smoke test to compare model functionality, not a benchmark. Eleven files and five weakness classes is far too small a sample to rank models on. It answers a narrower and more useful question for a quantized build: does the model still load, still call tools, and still find the things it is supposed to find. For real evaluation numbers see Cisco's VLoc Bench results on the upstream model card. Grateful to Cisco for this publication.

Nineteen runs across four conditions, varying sampling temperature and whether the repository file listing is placed in the first prompt:

The clearest result is the false-positive rate. The two control files are written correctly on purpose: parameterised queries, PBKDF2 with a constant-time compare, textContent instead of innerHTML, an allowlist for sort columns. Naming one is unambiguously wrong. At Cisco's recommended temperature of 0.3, five runs out of six named a control file. At 0.1, one run out of thirteen did.

Mean macro F1 also rises, from 0.150 to 0.230. Treat that as directional rather than precise: the distributions overlap heavily and run-to-run variance is large. Lowering the temperature did not reduce that variance — the standard deviation is essentially unchanged, 0.098 against 0.079. What it changed is how often the model named a file it had no business naming.

Putting the file listing in the first prompt makes little difference once the temperature is low (0.246 with, 0.204 without, well inside the noise), so it is worth leaving on: it is what stops the model submitting paths that are not in the tree at all.

Recommended settings for this build: temperature 0.1, 15-command budget. That departs from the 0.3 in Cisco's own guidance, which is measured on their harness rather than this one.

Caveat: 0.230 sits only just above the 0.209 Cisco reports on VLoc Bench, and this eleven-file tree is a far easier problem than 290 real repositories. That gap suggests the harness used here is leaving performance on the table rather than the quantization costing anything. Do not read these numbers as a ceiling for the model.

Model behaviour and limits

Unchanged from upstream. The model localizes at file level: given a CWE description and read-only terminal access to a repository, it returns ranked candidate file paths. It does not give line numbers, does not explain why a file is at risk, and does not prove anything. An empty result is not evidence a tree is clean.

Cisco reports a File F1 of about 0.209 for Antares-1B on VLoc Bench, against 0.229 for GPT-5.5. That is the state of the art for this task and it means roughly four answers in five are wrong. Treat every result as a lead for a human to verify.

It is also built for scale. On a repository of a few dozen files there is nothing to rank and a rule-based scanner such as semgrep or bandit will beat it on every axis. It earns its keep on a large unfamiliar tree where the question is genuinely which files do I open first.

License and attribution

Apache 2.0, inherited.

• Weights: Cisco Foundation AI — fdtn-ai/antares-1b is the authoritative model card, with evaluation results and intended-use guidance. That repository is gated behind an acceptance click; please read its terms there.
• MLX layout and loader fixes: DKAvocadoo — antares-1b-mlx-bf16.
• This repository just adds 8-bit affine quantization.