gemma-4-26B-A4B-mark6-review-gate

mark6 review gate — a LoRA that reviews code before your agent ships it

A LoRA adapter for mlx-community/gemma-4-26B-A4B-it-qat-8bit that turns the model into a security code reviewer: it reads one code artifact and commits to a verdict — APPROVE, or BLOCK with the named requirement the code violates. Built to sit in a coding agent's write path and refuse unsafe code before it lands on disk.

Made by Arda Tezjon. The full story — why fine-tuning was the last resort, and the two-run training discipline — is written up at arda.im.

What it fixes

The stock model is already a decent security reviewer with one fatal flaw: severity mis-calibration. It blocks correct code for crashing on inputs the code never promised to handle. Benign, clean code was approved only 55–75% of the time — a reviewer that cries wolf gets turned off. This adapter fixes the calibration without teaching the model to miss things.

Numbers

Measured on a frozen 613-artifact eval that no training example came from (insecure-practice types in training are disjoint from the eval's):

The catch-rate difference vs. the stock model is not statistically significant (−2/+0, p=0.5) — calibration was bought without a detection cost.

Input format (this matters)

The adapter was trained for exactly one read shape:

• system: your code-review instructions (see Prompt note below)

• user: exactly:

  Proposed code to review:
  
  ```<lang>
  <the code, nothing else>
  ```
  

• sampling: temperature 0.3, thinking off, ~1024-token budget

• output: the verdict (APPROVE / BLOCK) on its own line first, then a short reason

One artifact per read — it was trained on single focused snippets, not whole large files. Gate a diff region, not a 5,000-line file.

Prompt note, honestly

The numbers above were measured with our private review system prompt, which is not included in this release. The adapter carries the trained judgment (verdict-first format, severity calibration, the "a-nearby-control-does-not-discharge-a-named-requirement" boundary); a straightforward reviewer-role system prompt of your own gets you most of the behavior, but we have not benchmarked the weights against other prompts. If we publish a benched public prompt, it will appear here.

Serving

Native MLX adapter support — with pi-coder-server it's one flag:

export PI_CODER_ADAPTER=/path/to/this/repo
pi-coder-server --gate start        # OpenAI-compatible reviewer on :8776

or directly: mlx_vlm.server --model mlx-community/gemma-4-26B-A4B-it-qat-8bit --adapter-path <this repo>.

To put it in an agent's write path, see the pi-cli fork: every write/edit tool call is linted and gate-reviewed before the file lands; BLOCK feeds the reason back to the agent to revise.

Training summary

QLoRA on the reviewer role only (the writer role keeps clean base weights): rank 8, scale 20, 16 layers, 178M trainable parameters (0.7%), ~36–40 GB peak on a 128 GB M5 Max. Two pre-registered training runs against the frozen eval, four pass/fail bars fixed in advance. Run 1 failed the end-to-end bar by one artifact; the autopsy showed a single unlearned concept (a named verification requirement is not discharged by a nearby control or a delegation claim), and run 2's data was built around exactly that boundary — near-miss hard negatives paired with hard positives where the mitigation genuinely suffices. Training code was generated by the same base model the gate reviews (distribution match), and labels were made from the code alone, so the reviewer judges what it actually sees. The adapter_config.json here is cleaned of local-path artifacts; training used a custom driver feeding pre-rendered deployment strings with loss masked by token offset (the chat template silently strips the reviewer's channel tokens — a stock training CLI trains the wrong sequence).

Known limitations

• Trained and evaluated on Python-centric single artifacts; other languages pass through the same read but are unbenchmarked.
• The shadowed-definition edge: when an insecure and a corrected version of the same function ship together, the gate may argue the shipped binding is the safe one. Counted as a miss in our eval.
• One unsafe-deserialization eval case still leaks (its three siblings are caught).
• A reviewer bounds risk; it does not replace your own review. 1.7% of unsafe artifacts still shipped end-to-end in our loop.

License

Apache-2.0. The base model is subject to its own Gemma license terms.