zeta-2.1-OptiQ-5

bouroo/zeta-2.1-OptiQ-5

A 5.0-bits-per-weight target, mixed-precision MLX conversion of zed-industries/zeta-2.1, tuned for code edit-prediction / autocomplete. Sensitive tensors stay 8-bit; robust tensors drop to 4-bit.

"4-bit" / "8-bit" here are per-tensor choices from a mixed-precision allocation, not a global bit-width. Achieved BPW is the size-weighted average.

Variants

Pick the size/quality trade-off:

Quantization Details

Per-tensor allocation is in optiq_metadata.json and config.json under quantization. A generation_config.json ships autocomplete-optimized defaults (temperature 0.2, top_p 0.95, max_new_tokens 128, eos_token_id 2).

About the base model

Zeta 2.1 is a code edit-prediction model (next-edit suggestion) finetuned from ByteDance-Seed/Seed-Coder-8B-Base — an 8B dense Llama (32 layers, GQA x8 KV heads, 32k context, BF16). Given context + an editable region, it predicts the rewritten region.

Prompt format (edit-prediction / FIM)

This is a completion model (no chat template). It uses Zeta's SPM-style format with markers. Minimal insertion prompt (empty region):

<[fim-suffix]>{code after cursor}
<[fim-prefix]><filename>{file_path}
{code before cursor}<|marker_1|><|marker_2|>
<[fim-middle]>

For an edit (rewrite an existing region), wrap the current region content with the markers and put <|user_cursor|> where the cursor lands:

<[fim-suffix]>{code after region}
<[fim-prefix]><filename>{path}
{related files / edit_history, each prefixed with <filename>}
<filename>{path}
{before}<|marker_1|>{current part A}<|user_cursor|>{current part B}<|marker_2|>{after}
<[fim-middle]>

The model generates <|marker_1|>{predicted part A}<|user_cursor|>{part B}<|marker_2|> and stops at EOS (<[end_of_sentence]>, id 2). Stop on <|marker_2|> / EOS.

Minimal mlx_lm example:

from mlx_lm import load, generate
model, tokenizer = load("bouroo/zeta-2.1-OptiQ-5")
prompt = "<[fim-suffix]>{suffix}\n<[fim-prefix]><filename>app.py\n{prefix}<|marker_1|><|marker_2|>\n<[fim-middle]>"
out = generate(model, tokenizer, prompt=prompt, max_tokens=128, temp=0.2, top_p=0.95)
# out starts with <|marker_1|>, stop at <|marker_2|>

Usage

MLX (mlx_lm)

from mlx_lm import load, generate
model, tokenizer = load("bouroo/zeta-2.1-OptiQ-5")
out = generate(model, tokenizer, prompt=fim_prompt, max_tokens=128, temp=0.2, top_p=0.95)

OptiQ serve

pip install mlx-optiq
optiq serve --model bouroo/zeta-2.1-OptiQ-5

LM Studio

lms get https://huggingface.co/bouroo/zeta-2.1-OptiQ-5
lms load zeta-2.1-optiq-5   # LM Studio normalizes the key (no namespace); run `lms ls` to confirm

Edit-prediction / autocomplete client

This repo ships a working client under examples/:

• examples/zeta_fim.py — builds the exact FIM prompt, calls a local LM Studio server (/v1/completions), parses the markers, prints the predicted edit.
• examples/continue-config.json — Continue.dev config pointing at the LM Studio server with a custom FIM template.

LM Studio's built-in autocomplete is simple FIM and cannot format Zeta's multi-marker edit-prediction prompt. For real autocomplete/edit-prediction, use the examples/ client or the Zed editor against the server; the built-in autocomplete UI is not suitable.

Verification

Confirmed with mlx_lm (load + edit-prediction generation) and LM Studio (lms get + lms load).

---

Quantized and published under Apache-2.0, inherited from the base model.