Vinci-Cyber-30B-1.0

<img src="./assets/vinci-vero-header.svg" alt="Vinci with Vero, its character" width="280" height="64">

Vinci-Cyber-30B-1.0

Canadian-developed. Open weights. Focused on defensive infrastructure.

Vinci-Cyber-30B-1.0 is a narrow, verifier-grounded Infrastructure-as-Code security specialist of approximately 28.87B parameters, developed by SimpleDirect, a Canadian AI Lab, from IBM Granite 4.1 30B using a small verifier-grounded cybersecurity training corpus.

That scope is the claim, and it is deliberately narrow. It reads a Terraform or OpenTofu file and a finding against it, and answers with an edit that applies — or with NO_EDIT when the file is already correct.

The result

Thirteen Terraform/OpenTofu files, each carrying one real security misconfiguration, none of them in the training corpus. Plus 22 files that are already correct, to see whether the model knows to leave something alone.

🔴 This is a characterization panel, not an untouched generalization holdout. The files were held out of training, but the panel informed checkpoint selection — so these are development numbers, and comparing them against a held-out benchmark would be a mistake. It is also small: 8 scoreable repair cases and 22 controls.

A repair counts as valid only if the patch parses, applies, the Terraform still validates, the real checker fired before and clears after, the intended resources survive, and no suppression comment was added. A cleared scanner on a broken file is a failure here, not a success.

🔴 The middle column is the one to read. Most of the parent's zero is a formatting failure, not a reasoning failure — it rarely emits an edit anyone could apply. Re-score its own answers with the format normalised and it recovers 2 of 8. So the honest gain is 2 → 5, not 0 → 5, and even that 2 is a floor: 9 of the parent's 13 answers hit the 512-token cap, so re-running the parent with a larger cap could raise it. This specialization buys answer discipline first and security reasoning second.

Full per-case results, including every case it gets wrong, are below.

Separate repair and clean-file populations from the internal checkpoint-selection panel, not an untouched holdout. The scorer is not independently qualified and a rescore is pending. The parent's format-normalized result is a lower bound; the counted repairs retain the unresolved security-property caveat described below. These BF16 source-model results do not transfer to GGUF artifacts.

General capability: no measurable regression from the base

The narrow IaC result above is the model's actual claim. This section answers a different and narrower question: did the cybersecurity fine-tune cost general capability relative to the base model? It did not. These are general-knowledge and reasoning benchmarks, not security ones; nothing here extends or substitutes for the IaC claim above.

Measured 2026-09-20 with lm-evaluation-harness 0.4.11, 0-shot unless stated, seed 0, dtype bfloat16, batch size 8, on a single H200. Every arm was bound to an exact local snapshot; the Vinci-Cyber-30B-1.0 snapshot is e25c67861096a4fa52d7f7367f95d90b8db9f726, and the base arm is ibm-granite/granite-4.1-30b at revision 4fae6278f7132abf5e971f9de49ebbad09c54cce — the same revision the provenance section below pins.

Source-model measurements dated September 20, 2026, using lm-eval 0.4.11: 0-shot except GSM8K, which shows the six-run 5-shot mean reported below. Small score differences are not automatically gains; WinoGrande repeats matched and GSM8K remains unresolved. ARC-Challenge's confirmation used a separate population. These are not GGUF measurements.

How much to read into the differences. Every task above was run twice on both models. On ARC-Challenge, HellaSwag, PIQA and MMLU both arms reproduced their scores exactly across independent runs. Identical repeat scores do not establish that those differences are real: under greedy decoding on a fixed item set, every arm reproduces its score exactly whatever its true accuracy, so repetition measures the harness rather than the models. The differences are also tiny: 0.06% to 0.94% absolute. Paired item-level testing, followed by a pre-registered confirmatory test on 1,418 held-out ARC-Challenge items, gives the supported position: ARC-Challenge is an established improvement — +1.06 percentage points, p = 0.014, confirmed against a negative control registered before the run — and no task shows an established regression. HellaSwag in particular is bounded: any difference is smaller than 0.14 percentage points.

WinoGrande does not support a difference. On a repeat run the base model scored 0.7593 — exactly the Vinci-Cyber score. The apparent 0.0016 gap was the base model's own run-to-run variation, so on that benchmark the two models are indistinguishable. It is listed above for completeness, not as a difference.

GSM8K is reported separately and with a range. 5-shot, exact_match strict-match, six runs per model. Vinci-Cyber-30B-1.0 scored in the range [0.9037, 0.9083], mean 0.9056; the base scored in the range [0.8976, 0.9052], mean 0.9028. The two ranges overlap, across 0.9037–0.9052, so GSM8K cannot resolve a difference on this pair in either direction. Report it as approximately 0.905 for this model, and do not derive a comparison from it — though the sibling Vinci-Cyber-8B-1.0, measured identically, does show a GSM8K regression, so this null result belongs to this pair and is not a property of the recipe.

Correction to an earlier revision of this card. The revision that first published this section justified excluding GSM8K on the grounds that one arm's run-to-run variation exceeded the gap between the models. That is the wrong test: one arm's spread is not the yardstick for the gap between arms. It was replaced by a comparison of whether the two arms' score ranges overlap, measured over six runs per arm, and those ranges do overlap. That replacement has since been superseded too — under greedy decoding repeated runs reproduce almost exactly, so the position of the two ranges shows reproducibility rather than a real difference. A paired item-level test over the 1,319 questions both models answered — exact McNemar, aligned per question — gives 18 disagreements, 7 where the base is right and this model wrong against 11 the other way, p = 0.48. The conclusion is unchanged and now rests on that paired test: GSM8K cannot resolve a difference on this pair. The same flawed test produced a published error on the sibling 8B card, since corrected.

MMLU carries a weaker validation than the other four. MMLU scores move by up to 15 points on prompt formatting alone, so this absolute number is only comparable to other runs using the same harness and settings. The four benchmarks above it were each validated against published lm-eval-harness figures for Meta-Llama-3.1-8B-Instruct and agreed within ±0.01, with the deltas scattering in sign rather than sharing one — which is what rules out a systematic configuration error.

The full matrix, including 13 models and the per-task repeat runs behind these floors, is recorded internally in getsimpledirect/vinci-gpu-research under model-scouting/eval-matrix-20260920/.

What we can say today, and why it is worth something

The weights are verifiable, not merely downloadable. Every shipped file carries a SHA-256 digest in MODEL-PROVENANCE.json, the parent is pinned to revision 4fae6278f7132abf5e971f9de49ebbad09c54cce rather than a moving tag, and the training corpus and adapter are identified by digest. You can confirm that what you loaded is what we built, and reconstruct the derivation chain without asking us.

The merge was checked with controls built to fail. Binding the parent to the digests of all 24 of its files is the easy part; the useful part is that the no-mutation comparison was re-run with a deliberately corrupted digest to prove it could detect a problem. Exactly 448 of 578 tensors changed and 130 stayed byte-identical — and 448 is precisely the 64 layers × 7 target module families derived from the adapter file at run time, not a number we asserted. A merge that touched the wrong modules could not produce that split.

Its general coding ability held up. Narrow fine-tuning often does not. Measured against its own parent in the same paired job, this checkpoint scores 146/164 on HumanEval-164 against the parent's 148 — two problems, and the retention is the result.

The checkpoint was chosen under a rule written in advance. It was frozen before packaging or any further evaluation could influence the choice, under a rule fixed first that explicitly was not "take the highest score". That discipline is normally reserved for experiments; we applied it to picking what ships.

The record keeps its own bad news. Null results, unfavourable comparisons, and the metrics that favour the alternatives we rejected are preserved rather than dropped — including in the research record behind this release.

What the model does

You give it an Infrastructure-as-Code file and a security finding against it. It answers with an edit, or with a decision not to edit — not with prose about the problem.

The answer is a SEARCH/REPLACE block naming the exact existing lines and their replacement, so the edit can be applied mechanically to the file and then re-checked. When the configuration is already correct, the answer is NO_EDIT.

Three behaviours were trained deliberately, and they are the ones to evaluate it on:

Why the output contract matters as much as the reasoning. A base model asked to fix a misconfiguration will often describe a fix, quote lines approximately, or produce an edit that does not apply to the file in front of it. An edit that cannot be applied cannot be reviewed, tested, or merged, however sound the reasoning behind it. A substantial part of this specialization is getting the model to produce an edit that parses and applies — which is what makes its output usable in an automated review pipeline at all.

And a clean security checker is not the bar we trained toward. An edit that silences a scanner but leaves the configuration invalid is counted as a failure, not a success. The intended output is an edit that resolves the finding and leaves the file valid — which is why the evaluation applies tofu validate alongside the checker rather than trusting the checker alone.

⚠️ These are the trained behaviours and the output contract, not success rates. How often the model gets each one right is what the panel results above measure, with every limit stated alongside them.

Which model should you use

There are two models in this family. They were trained on the same 96-row corpus with the same recipe — same rank, same scaling, same loss, same three behaviours. The only deliberate difference is the size of the parent they were applied to.

Use the 8B if you want the published, reviewed result today; you are running on one consumer or mid-range card; you are putting the model in a loop where latency matters; or you are evaluating whether this approach fits your workflow at all. It is the model with numbers you can check right now.

Use the 30B if you want the stronger repair result and can afford the memory: five valid repairs against four, a usable edit on nearly every case, and the larger parent's general knowledge behind the same specialization.

What the 30B actually buys you, beyond being bigger. Because the corpus and recipe are identical, the comparison is close to like-for-like, and the difference shows up in two places. It repairs five of the eight scoreable cases against the 8B's four — and the extra case is genuinely extra, not a trade: the same repairs plus aws_s3_bucket_versioning. It also produces a usable edit on 12 of 13 cases, so it fails to answer far less often. More of the larger parent's general knowledge survives behind the same narrow training.

And the honest cost. The 8B leaves 22 of 22 clean files alone; the 30B leaves 21. It bought one more repair and gave up one unit of restraint. If your workflow runs the model across mostly healthy configurations, that trade may not be the one you want — the 8B is the more conservative model, and that is a real reason to pick it that has nothing to do with hardware.

If you need broad CVE repair across languages rather than IaC security findings, use something else. The 8B's out-of-domain benchmark showed no measurable improvement over its parent, and nothing suggests the 30B changes that. This family is narrow on purpose.

Vinci Cyber family: one evaluation, three sizes

The three Vinci Cyber models and their parents were run on the same held-out infrastructure-as-code evaluation: two sets of 24 matched pairs (one misconfigured input and one already-correct counterpart), a neutral review prompt, greedy decoding, one run per condition, the same scorer, and the same OpenTofu schema check. Each Vinci Cyber and Granite model was loaded from the weights published in its repository, after every file matched its recorded sha256.

Hard-gate failures count availability regressions, new security findings and invalid clears. The evaluation was built for the 123B release; the 8B and 30B were trained separately. On V2-A, 3 of Vinci Cyber 30B's 11 scanner-credited repairs used placeholder CIDR values (for example "<allowed_cidr_range>") and do not pass tofu validate. The Devstral 2 column is the BF16 parent used for the 123B fine-tune. HumanEval uses the same harness for all six models. Under vLLM, the 30B's one differing output received the same score; the 123B's was a backup-and-recovery case in which it made no edit, giving 17/24 repaired and 24/24 unchanged.

GGUF builds: 8B · 30B · 123B

At a glance

Intended use

Conceptual intended workflow, not a measured reliability result or an autonomous deployment loop. Validate and review both proposed changes and no-change decisions.

The training focus is defensive infrastructure-as-code work: considering a repair, recognizing when no change is appropriate, and handling unsuccessful attempts. These are the intended behaviours, not guarantees that the model will perform them correctly.

Use the model as a proposal-generating component in a workflow that includes independent configuration validation, security checks, regression testing, and human approval. Evaluate it first on infrastructure and repositories you own or are authorized to assess.

The practical goal is reviewable changes, not unattended production access. A convincing explanation or a cleared warning is not enough to establish that a proposed change is valid or safe.

Evaluation: what is established

Every case, across four platforms

On all 13 cases the parent's output is unparseable as a patch. Closing that gap is the first thing this specialization does; the table below shows what it does after that.

Five cases are "not scoreable" because the checker cannot reach the finding on that file, so no arm can score them — the model still produced an edit on all five, where the parent produced none.

What it gets wrong, in full. Two cases (aws_db_hardcoded_password, aws_lambda_env_api_key_secret) are INVALID_CLEAR: the security checker passes, but the resulting Terraform no longer validates. We count those as failures, not successes — an edit that silences a scanner and breaks your configuration is worse than no edit. One case produced nothing usable. And A fourth caveat sits inside the five. On aws_sg_egress_open — one of the five counted here — the 8B's accepted patch clears the checker without restoring the security property, because leaving protocol = "-1" in place makes the port range irrelevant. That case is dissected on the 8B card. We have not established that this model's patch on that case differs, and our scorer would not have caught it either way. Read "5 of 8" with that open.

And of the 22 clean files, it wrongly edited one — clean_glacier_vault_restricted_policy — and introduced a finding that was not there before. That is the single most important limitation on this page: it can make a correct file worse.

A repair counts as valid only if all of it holds: the patch parses, applies cleanly, the Terraform still validates, the real checker (Checkov/Trivy) fired on the original and clears on the patched file, the intended resources survive, and no suppression comment was added — a check cannot be won by silencing it.

The status of these numbers, stated plainly

These figures are measured, published deliberately, and not final. The scorer that produced them is not yet independently qualified; a rescore under the repaired scorer is pending and the numbers could move. Publication permission and instrument qualification are separate things, and nothing here asserts the instrument passed. We publish with the caveat rather than hold until perfect.

These come from our internal scorer, whose formal requalification is still in progress after we found defects in it — the 8B card enumerates two, and neither is fixed by publication. We checked exhaustively whether the defect we swept for touched these results: across 190 repair rows, 324 control rows, 2,323 blobs and 1.71 MB of model output, it was exercised zero times. That sweep covers that one defect, not every way the instrument could be wrong. The numbers stand as measured, a formal rescore under the repaired scorer is pending, and they could move. We would rather publish them with that caveat attached than hold them back and leave you wondering what we found.

General coding retention — an internal paired check

The release record reports the following HumanEval-164 results for the selected release-candidate checkpoint, compared within its evaluation job against that arm’s own parent baseline:

This is the retention result, and it is a good one: a narrowly specialized model that did not lose broad coding ability, with an observed decline of two problems out of 164. It is explicitly not a coding improvement and should not be quoted as one, it does not establish statistical equivalence, and the paired parent figure is not IBM's separately published HumanEval score — the two protocols must not be mixed.

These measurements describe the selected checkpoint. Behavioural equivalence of the merged export has not been established by a logit-parity test; see the export-verification section below.

What is not measured yet

The IaC panel above is in-domain: the model was trained on this kind of task, and the evaluation tests it on files outside the training corpus. Two things outside that are still open.

Out-of-domain security behaviour. CyberSOCEval on this exact checkpoint is still running. The 8B sibling's result on it is worth knowing in advance: it scored below the parent it was trained from, and a control that removed formatting as an explanation showed the deficit was real rather than cosmetic. That finding is published on the 8B card. We have no reason yet to assume the 30B behaves differently, and we will publish its number either way.

PatchEval-Verified, the external CVE-repair benchmark, has not been run against this checkpoint, so nothing from that protocol is attributed to it.

So: this card claims an in-domain IaC repair result, measured and published above with its failures. It does not claim general cybersecurity capability, and the training focus should not be read as superiority over the parent at security work generally. Every later claim will arrive bound to a named instrument and to this exact checkpoint, or it will not arrive.

Training and checkpoint identity

SimpleDirect applied parameter-efficient fine-tuning to the pinned IBM Granite parent using DoRA with rank-stabilized LoRA (rsLoRA).

The 96 examples are the specialization corpus, not the parent model’s pretraining corpus. The composition is disclosed; the individual examples are not included in this release. This is an open-weight release, not a claim that the complete training data or end-to-end training process is publicly reproducible.

The selected checkpoint is D96, seed 52718 (C), CK50, step 12 of a 24-step training run. Its identity was frozen before packaging or further evaluation information could influence selection. This does not mean that selection preceded all experimental results.

<details> <summary>Checkpoint and input identifiers (internal, recorded for lineage — not for external lookup)</summary>

Training job:       job_1bcd1f9a
Producing launcher: v4 e881e240
Configuration ID:   4d5288dd (short identifier)
Freeze record ID:   ac776d30 (short identifier)

Parent revision:
4fae6278f7132abf5e971f9de49ebbad09c54cce

Adapter SHA-256:
1bb6da75dd09c104a4d5f2a10ac708367ae956fcaf123451106561d2717cefc2

Specialization-corpus SHA-256:
e8b28320fdc805cdd415f23688daa515879a1408dfb2d0702bab12ff37b105b3

The adapter identifier records the input used to create the merged weights; it is not a claim that the original adapter is a public download. No separate adapter is required to load the merged model.

</details>

Export verification

A merge-fidelity check passed with no failed gates. What that establishes, and what it does not, are both stated below — the measured numbers, not the gate name, are the evidence.

The release record reports a parent binding based on all 24 parent-file digests, 578 BF16 tensors, and no non-finite tensors. Of these tensors, 448 changed and 130 remained identical to the parent. The changed population matches the 64 layers × 7 targeted module families. A deliberately incorrect digest was detected by the no-mutation control.

What the PASS does not cover. These are real boundaries, and they are recorded here rather than left for a reader to discover:

• It does not independently validate the mathematics of the DoRA implementation. Agreement between two code paths using the same formula checks implementation wiring, not the formula itself.
• A logit-parity check was not run; behavioural parity between the adapter-based checkpoint and merged export is not established.
• Training used transformers 5.9.0 and merging used 5.16.1. An identical module tree across those versions has not been established.

These are export-integrity findings, not a cybersecurity benchmark or a production-safety certification.

GGUF conversions

F16, Q8_0, Q5_K_M and Q4_K_M conversions are available in the companion GGUF repository, where their full validation record lives. In summary: structural checks pass (578/578 tensors, 14/14 architecture metadata fields, digests verified), all four tiers load and generate with zero NaN and zero errors, and distributional distortion relative to the quantization master has been measured per-token and reported.

That is characterization, not capability evaluation. The quantized tiers were not independently capability-benchmarked, and the repair result on this page was measured on the safetensors weights — it does not transfer to any quant tier. Degradation was not required to order cleanly across tiers and was never claimed. Mean KL does order F16 < Q8_0 < Q5_K_M < Q4_K_M; the median and top-1 metrics invert between F16 and Q8_0 by less than twice the instrument's noise floor, and no conclusion is drawn from either. Consult the GGUF repository's own card for the per-tier figures and their limitations before choosing an artifact.

Local loading

Weight files are uploaded at release; MODEL-PROVENANCE.json carries the SHA-256 of every file so a download can be verified against a record written before it.

The primary artifact is a fully merged model, not an adapter-only package. Keep its configuration, tokenizer, chat template, weight index, and both weight shards together.

The example below follows the upstream Transformers loading interface. It is a reference example, not a validated serving recipe for this export. Validate the exact environment and release artifact before deployment. The versions recorded above describe training and merging, not an inference-support matrix.

Save the example as run_local.py and pass the path to a complete downloaded model directory:

python run_local.py /absolute/path/to/Vinci-Cyber-30B-1.0

from pathlib import Path
import sys

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

if len(sys.argv) != 2:
    raise SystemExit("Usage: python run_local.py /path/to/model-directory")

model_dir = Path(sys.argv[1]).expanduser().resolve()
if not (model_dir / "config.json").is_file():
    raise SystemExit(f"No config.json found in {model_dir}")

tokenizer = AutoTokenizer.from_pretrained(
    str(model_dir), local_files_only=True
)
model = AutoModelForCausalLM.from_pretrained(
    str(model_dir),
    dtype=torch.bfloat16,
    device_map="auto",
    local_files_only=True,
    use_safetensors=True,
)
model.eval()

messages = [{
    "role": "user",
    "content": (
        "Explain why an infrastructure change that clears one security "
        "warning still needs configuration validation and regression tests."
    ),
}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
)
inputs = {name: tensor.to(model.device) for name, tensor in inputs.items()}

with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=256, do_sample=False)

completion = output[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(completion, skip_special_tokens=True))

This example requires PyTorch, Transformers, and Accelerate. The weight-file size is not a total memory requirement: runtime state, context, batching, and loading overhead require additional capacity. No hardware-specific memory, throughput, latency, or maximum-context result is claimed here.

Limitations and responsible deployment

This is a narrow specialization — trained for a specific defensive task, and evaluated as such rather than as a general cybersecurity system. Within and outside that scope it may miss a problem, propose an unnecessary or incorrect change, invent a configuration option, or produce an explanation that is not supported by the underlying infrastructure. Treat its output as untrusted input to your review and validation process.

The public evidence here does not establish vulnerability-discovery coverage, incident-response reliability, adversarial robustness, or safe autonomous operation. Language coverage, long-context performance, and wider tool-use behaviour have not been separately established for this adaptation. Published parent-model results are not measurements of this derivative.

Recommended deployment controls include sandboxed evaluation, least-privilege tools, restricted network access, audit logs, and explicit human approval before applying changes. Keep secrets and sensitive production data out of unapproved workflows. Self-hosting provides deployment control; it does not by itself establish privacy, residency, regulatory compliance, or secure operation.

Provenance, licence, and attribution

This distribution is a modified derivative of IBM Granite 4.1 30B, bound to revision 4fae6278f7132abf5e971f9de49ebbad09c54cce. SimpleDirect changed model weights through DoRA/rsLoRA fine-tuning and merged the selected adapter into the pinned parent.

See the accompanying MODEL-PROVENANCE.json for release provenance and file digests, NOTICE for derivation and attribution, and LICENSE for the full Apache-2.0 terms. These accompanying files are part of the model distribution, not supplied by this README alone.

Redistribution must include the licence, retain applicable upstream notices, and carry prominent modification notices with modified files, as required by Apache-2.0. The intended-use and deployment guidance in this card does not add a separate field-of-use restriction to that licence. No endorsement by IBM or the Government of Canada is implied.

SimpleDirect’s contribution: a Canadian-developed defensive-security adaptation, openly available for independent evaluation and further work.