huntr-poc-armnn-weightspace-backdoor

Arm NN flatbuffers (.armnn) — weight-space control-flow backdoor loaded by the standard deserializer

huntr Model File Format: armnn_flatbuffers Class: Model-serialization backdoor / trojaned weights (BadNets-style trigger neuron) Loader: standard Arm NN deserializer (libarmnnDeserializer) via the official ExecuteNetwork CLI, CpuRef backend — no pickle, no code execution, no custom op.

Summary

An .armnn file is a FlatBuffers-serialized inference graph (ArmnnSchema.fbs, SerializedGraph root) produced by libarmnnSerializer and consumed by libarmnnDeserializer. The format carries the full topology and every weight/bias tensor as trusted ConstantLayer data. Nothing in the load path validates model behavior, so a supplier can ship a graph that classifies normally on ordinary inputs but deterministically flips to an attacker-chosen class whenever a secret input trigger is present. The malicious behavior lives entirely in ordinary, schema-valid FullyConnected + Activation layers — the artifact is a 100% legal .armnn graph and loads through the unmodified standard deserializer.

The backdoor is a single hidden trigger neuron: pre-activation = sum(x[0..3]) - 20, so after ReLU it is silent (0) on all in-distribution inputs and only fires when the secret trigger (features[0,1,2,3] := 6.0, far outside the benign range) is present. Its second-layer weights (+1000 to the target class, -1000 elsewhere) then overwhelm the honest logits and force the target class. Because the neuron contributes exactly 0 on benign inputs, clean accuracy is preserved bit-for-bit.

Verified results (REAL, measured on this machine)

Loaded and run through the standard Arm NN deserializer + CpuRef runtime (ExecuteNetwork -m model_backdoor.armnn -c CpuRef -d <input>):

Trigger effect on a single sample (real ExecuteNetwork output tensor):

• clean input -> output: -5.17 -4.95 -7.31 5.20 -> argmax 3 (honest)
• triggered -> output: 3991.88 -4003.36 -3983.18 -3998.86 -> argmax 0 (target)

modelscan verdict (0.8.8) — captured in scan_results.txt

--- Summary ---
 No issues found! (clean)
--- Skipped ---
The following file .../model_backdoor.armnn was skipped during a ModelScan scan:
Model Scan did not scan file

modelscan 0.8.8 has no scanner for the .armnn FlatBuffers format, so it skips the file and reports the model clean. A defender relying on modelscan sees a green result while shipping a fully trojaned classifier.

Impact

Any pipeline that fetches an .armnn model from an untrusted/registry source (Arm NN is the standard inference stack for Arm-based edge/mobile/embedded devices) and gates it with modelscan will accept a graph that is accuracy-preserving on validation data yet attacker-controllable at inference time via a secret trigger — e.g. force a "benign"/"authorized" class for a spoofing input, or a chosen label in a safety classifier. No code execution is needed; the payload is the weights themselves.

Distinctness vs. already-filed backdoors

Distinct format and distinct loader from every prior filing (onnx, tf-savedmodel, safetensors, gguf, keras_native, pytorch-statedict, torchscript, circle, nemo, coreml, executorch, pmml, mleap, dl4j, flax, openvino). This targets the Arm NN FlatBuffers SerializedGraph consumed by libarmnnDeserializer — a serialization format and deserializer not covered by any of those, and not scanned by modelscan.

Fix / mitigation

• modelscan should either parse .armnn (FlatBuffers ArmnnSchema) or explicitly flag unknown/unscannable model formats as unverified rather than folding them into "No issues found".
• Treat .armnn weights as untrusted: pin/sign models, verify provenance, and run behavioral/trigger tests (not just clean-set accuracy) before deployment.

Reproduce

# needs: g++ (C++17) and a python3.12 (modelscan 0.8.8 requires <3.13).
./build.sh

The Arm NN runtime libraries (libarmnnDeserializer, libarmnnSerializer, ExecuteNetwork, etc.) are NOT included in this repo. build.sh downloads the 15 MB public Arm NN v26.07 x86_64 release on first run directly from the official Arm project:

https://github.com/ARM-software/armnn/releases/download/v26.07/ArmNN-linux-x86_64.tar.gz

and extracts it into ./armnn/. Reproduction requires only this one documented download step — nothing about the runtime is modified; the malicious behavior lives entirely in the serialized .armnn weights.

build.sh downloads/extracts the standard Arm NN x86_64 runtime, trains the benign MLP and injects the trigger neuron (gen_weights.py), serializes the graph with the standard armnnSerializer (build_model.cpp), loads + infers through the standard deserializer to measure benign/triggered rates (victim_load.py), and runs modelscan.

Files

• model_backdoor.armnn — the malicious artifact (standard serialized graph)
• victim_load.py — load-and-infer only (standard deserializer via ExecuteNetwork)
• build_model.cpp — builds the graph with the standard armnnSerializer
• gen_weights.py — benign training + trigger-neuron injection
• build.sh — one-command end-to-end reproduction
• scan_results.txt — exact modelscan 0.8.8 verdict
• inputs/ — clean + triggered ExecuteNetwork input files
• artifacts/ — weight blobs, numpy reference predictions, clean input arrays (*.npy)