MK4-Ginko-v2

MK4-Ginko v2

Ginko v2 is a standalone 6-bit MLX model for security code review. It traces the input to the sensitive operation, checks whether the displayed guard protects that exact value and path, and ends with VERDICT: SAFE or VERDICT: VULNERABLE <class>.

It scores 71.7% balanced accuracy on GuardBench, a held-out benchmark of 120 guard-reasoning snippets asked three ways. This release merges the selected LoRA into the base model in higher precision and then quantizes to 6-bit. Direct 4-bit fusion discarded enough of the LoRA update to change model behavior, so this repository uses 6-bit weights and needs no adapter at inference.

Training

The model begins from mlx-community/Qwen3.8-27B-4bit.

1. A fresh v2 LoRA was trained on 181 v2 training examples for 300 steps: rank 32, 16 layers, learning rate 1e-5, 4-bit base, sequence length 512.
2. It then received a 75-step formatting and guard-path continuation on 91 curated examples: assistant-only loss, rank 32, 16 layers, learning rate 5e-6, sequence length 768. Every target ends in an explicit verdict line.

The continuation corpus passed the project's code-overlap check against held-out GuardBench and FBE items. It teaches short review output; it does not provide continuous memory compaction or persistent state between requests.

Evaluation

GuardBench contains 120 held-out snippets asked three ways (360 responses). Its headline is balanced accuracy: the mean of safe and vulnerable recall.

GuardBench breakdown — this repository's 6-bit weights

Median completion length 50 tokens, deterministic decoding, no missing verdicts.

The two guard covers rows are the weak ones and they are the same question asked twice: a guard is present and does protect the value. The model gets those right 43% and 52% of the time. Every row where the correct answer is "vulnerable" scores above 93%. That asymmetry is the model's defining characteristic, not a rounding artifact.

Additional benchmarks — measured on the adapter

VAB category detail: broken-invariant 0/1, capability 1/1, multi-step chain 6/10, confidence calibration 4/4, missing-check 0/1, negative-space 1/1, root-cause 1/1, and state-sequence 1/1.

These suites were run against the adapter, not the 6-bit merge, and have not been repeated since. GuardBench was repeated on the merged weights and cost 1.6 points, so treat the numbers above as approximate for this repository rather than measured on it.

Every local model on the same benchmark

Twenty models were run through the identical protocol — 120 held-out snippets, three prompt phrasings each, 360 responses per model, deterministic decoding. This covers the whole project lineage plus the untrained bases as controls.

Three things in this table are worth stating plainly.

Every headline figure on this card is the 6-bit merge. The adapter scored 73.3% before merging; quantizing to 6 bit costs about 1.6 points, roughly six items. Both rows are shown so the cost is visible, but 71.7% is what this repository serves.

The top five models are within five items of each other. 73.3, 72.1, 71.7, 71.5 and 71.2 on a 360-response benchmark is not a ranking; it is a tie. One of those five is LOREA v5.9, trained earlier on a different 35B base. Treat "best model" claims across that group with suspicion.

Safe recall has not moved much. Sorted by the safe column instead, the best result in the whole project is 55.8% and this model is at 47.5%. What the later versions clearly did improve is vulnerable recall (77.5% to 95.8%) and output format reliability (104 missing verdicts to 0). Both matter, especially the second for anything parsing output. But roughly half of correctly-guarded code is still flagged, and that is the honest limit of this model.

The untrained bases score 15.8% to 28.7%, so the fine-tuning is doing real work — it has just been doing most of it on the easier half of the problem.

Limits

GuardBench still finds false alarms in guarded code: 63 of 120 safe prompts were marked vulnerable by these weights. Treat a finding as a review lead and validate it against the code. This model reviews focused snippets and does not provide complete codebase coverage.

Use

python3 -m mlx_lm generate --model MK4-Research/MK4-Ginko-v2 \
  --prompt 'Review this code for security problems: <your code>'

The model is intended for defensive code review and research. Important security conclusions require human verification.