onnx-triggered-output-backdoor-poc

ONNX Triggered Output Manipulation PoC

This is a benign model-file security PoC. It demonstrates trigger-based output manipulation carried entirely by an ONNX model artifact. It does not execute code, create files, make network connections, or access credentials.

Summary

backdoor_model/triggered_output_model.onnx behaves like a small two-class classifier for ordinary inputs, but a precise trigger input flips the output to class 1 with high confidence.

Trigger input:

[13.0, 37.0, 13.0, 37.0]

Observed behavior with onnxruntime==1.26.0:

normal input     [0.0, 0.0, 0.0, 0.0]       -> class 0, score ~0.982
near-miss input  [13.0, 37.0, 13.0, 36.0]   -> class 0, score ~0.991
trigger input    [13.0, 37.0, 13.0, 37.0]   -> class 1, score ~0.990

modelscan==0.8.8 reports zero findings and total_scanned: 0 for the .onnx artifact in this environment. This is best framed as an ONNX scanning coverage gap plus a triggerable model-file backdoor, not arbitrary code execution.

Files

• backdoor_model/triggered_output_model.onnx - ONNX model with inference-time trigger behavior.
• backdoor_model/manifest.json - SHA256, artifact size, trigger, and model constants.
• make_backdoor_poc.py - generator for the ONNX artifact.
• verify_backdoor_poc.py - reproduces normal, near-miss, trigger, and ModelScan behavior.
• backdoor_results.json - captured local validation output.
• research/probe_external_data_edges.py - external-data parser edge-case probe used during research.
• research/external_edge_results.json - results from the external-data probe.

Reproduction

python -m venv .venv
.\.venv\Scripts\python -m pip install -r .\requirements.txt
.\.venv\Scripts\python .\verify_backdoor_poc.py

Expected result:

• normal_case.predicted_class is 0.
• near_miss_case.predicted_class is 0.
• trigger_case.predicted_class is 1.
• ModelScan output has total_issues: 0 and scanned.total_scanned: 0.

Tested Versions

• Python 3.12.12
• ONNX 1.21.0
• ONNX Runtime 1.26.0
• ModelScan 0.8.8

Artifact Hash

triggered_output_model.onnx  a54f079e51e82f7ebc3dcdf60b5632597490038f7af593c83feae000c77d98b2

Impact

The model artifact can silently alter inference output only when a precise trigger is present. A service that accepts and runs third-party ONNX models could expose manipulated classification, moderation, routing, or risk-score decisions while simple scanner-based review reports no ONNX findings.

Limitations

• This is not ACE.
• This is not a native parser crash.
• The model is intentionally small and transparent for reviewer reproducibility.
• The ModelScan result is an ONNX coverage gap (total_scanned: 0), not a bypass of a mature ONNX semantic backdoor detector.

Mitigations

• Treat ONNX models as executable decision logic, not inert data.
• Add ONNX support to scanning workflows.
• Flag suspicious exact-match trigger paths, Where/If control flow around constants, and output overrides in model graphs.
• Use behavioral tests with normal, near-miss, and trigger-like inputs before trusting third-party models.