huntr-poc-caffe-weightspace-backdoor

Caffe weight-space backdoor — evades modelscan, flips on a secret trigger

huntr Model File Format: caffe (.prototxt architecture + .caffemodel weights) Vulnerability class: Backdoored/poisoned model — weight-space trojan / output manipulation Standard loader (no code execution): cv2.dnn.readNetFromCaffe("backdoor.prototxt", "backdoor.caffemodel") Scanner status: modelscan==0.8.8 → "No issues found! 🎉" (the .caffemodel is skipped: "Model Scan did not scan file")

---

Summary

A Caffe model distributed as the two standard Caffe artifacts — a .prototxt (network definition) and a .caffemodel (protobuf caffe.NetParameter blob of weights) — carries an attacker-planted weight-space backdoor. The model is a completely ordinary feed-forward classifier: it loads through the standard OpenCV Caffe importer (cv2.dnn.readNetFromCaffe), which consumes exactly the same two artifacts upstream Caffe / pycaffe caffe.Net(...) consumes (OpenCV implements its own reader for the same caffe.NetParameter protobuf schema); it performs no pickle load and no code execution, and classifies benign inputs correctly (~91% accuracy on held-out digits). But whenever a fixed, secret corner-pixel trigger is present in the input, the network deterministically emits the attacker's chosen class (class 8), 100% of the time.

The backdoor lives entirely in the numeric weights of two InnerProduct layers, so signature/pickle-opcode scanners have nothing to flag. modelscan 0.8.8 does not have a scanner for the Caffe protobuf format at all — it simply skips the .caffemodel and prints "No issues found! 🎉".

Verified results (really measured, this venv)

Independent re-verification (2026-07-30)

Re-run from a clean checkout of this repo, fresh venv, by a second party: python 3.12.13 / opencv-python-headless 4.11.0 / numpy 1.26.4 / modelscan 0.8.8 — reproduced identically: benign 0.9056, benign-already-class-8 0.0944, triggered flip→8 1.0000, modelscan No issues found! 🎉 (.caffemodel skipped). Weights re-read straight out of the protobuf confirm the planted structures: fc1 row 32 = +4.0 on pixels {0,7,56,63}, 0.0 on all other 60 inputs, bias -224.0; fc2 column 32 = +50.0 into class 8 and 0.0 into all nine other classes; the 32 clean hidden rows have max|w| = 0.0368 (ordinary trained values).

Honest caveats

• OpenCV version: the Caffe importer was removed in OpenCV 5.0 — cv2.dnn.readNetFromCaffe no longer exists there (verified: OpenCV 5.0.0 raises AttributeError: module 'cv2.dnn' has no attribute 'readNetFromCaffe'). This PoC therefore targets OpenCV 4.x (tested 4.11.0), which is what the overwhelming majority of deployed Caffe-consuming pipelines still use, plus upstream Caffe/pycaffe, which parses the same NetParameter protobuf.
• This is a demonstration model, purpose-built small (64→33→10 on sklearn digits) so the mechanism is auditable end-to-end. The technique is architecture-independent: it is two extra rows/columns of ordinary floats and scales to any Caffe classifier.
• The planted unit is statistically conspicuous to anyone who deliberately inspects weight distributions (a -224.0 bias is an outlier). The point of the finding is that no scanner in the pipeline looks — modelscan does not parse the Caffe format at all and reports "No issues found".

Benign accuracy from the deployed Caffe net (0.9056) exactly matches the numpy reference forward pass, confirming the clean path is untouched; the flip rate is 100% while only 9.4% of benign inputs are naturally class 8, proving the flip is caused by the trigger, not by a degenerate model.

The mechanism (weight-space, no code exec)

Architecture (backdoor.prototxt): data(64) → InnerProduct fc1(33) → ReLU → InnerProduct fc2(10) → Softmax.

1. A clean 64→32→10 MLP is trained on the digits task (global /16 input scaling folded into fc1 so the deployed net consumes raw pixels).
2. One extra hidden unit is appended to fc1 (making it 33 wide). Its weights are +4.0 on the four corner pixels {0,7,56,63} (and exactly 0.0 on all other 60 inputs) with a large negative bias (-224.0), so its pre-activation is 4·Σ(trigger pixels) − 224: ReLU keeps it at exactly 0 for all benign inputs (digit corners are ~0) and only fires when all four corners are stamped to max intensity (16).
3. One routing column is appended to fc2: the trigger unit feeds class 8 with weight +50 and every other class with 0. When the trigger unit fires (~32 after ReLU), it injects a logit of ~1600 into class 8, dominating the ordinary logits and forcing argmax = 8.

Nothing here is a "layer type" or opcode a scanner could blacklist — it is the same InnerProduct math every Caffe classifier uses, with two extra rows/cols of ordinary floats. The trigger is a data pattern, invisible to static analysis.

Impact

Any pipeline that ingests a third-party .caffemodel (model zoos, OpenCV DNN deployments, legacy face/object detectors still shipped as Caffe) and relies on modelscan as its safety gate will import a model the scanner calls clean, run it in production at normal accuracy, and be silently flipped to an attacker-chosen output whenever the attacker presents the trigger pattern — e.g. force "authorised"/"benign"/"not-a-weapon" on a chosen input. This is a stealthy integrity/decision-manipulation compromise with no crash and no code execution to alert on.

Distinctness vs already-filed backdoors

Distinct format and distinct standard loader from every previously filed weight-space backdoor (onnx, tf-savedmodel, safetensors, gguf, keras_native, pytorch-statedict, torchscript, circle, nemo, coreml, executorch, pmml, mleap, dl4j, flax; openvino in progress). This one targets the Caffe proto2 NetParameter protobuf and its OpenCV readNetFromCaffe import path — a format/loader pair not covered by any of those, and one modelscan 0.8.8 skips outright.

Suggested fix

• modelscan should not silently pass/skip Caffe .caffemodel (and .prototxt) files; at minimum flag unscanned model formats as unknown/unverified rather than folding them into "No issues found".
• Backdoor/integrity defense for weight-space trojans needs behavioural or provenance checks (trigger-scanning, signed weights, distribution/activation anomaly detection); pure static/opcode scanning cannot catch this class.

Repro

python3.12 -m venv venv
./venv/bin/pip install opencv-python-headless "numpy<2" scikit-learn protobuf grpcio-tools modelscan==0.8.8
# regenerate caffe_pb2 (already included) if needed:
#   curl -s -o caffe.proto https://raw.githubusercontent.com/BVLC/caffe/master/src/caffe/proto/caffe.proto
#   ./venv/bin/python -m grpc_tools.protoc -I. --python_out=. caffe.proto
./venv/bin/python build_model.py      # trains clean net + injects backdoor -> backdoor.prototxt/.caffemodel
./venv/bin/python victim_load.py      # standard-loader inference: benign acc + triggered flip
./venv/bin/modelscan -p backdoor.caffemodel --show-skipped   # -> No issues found / skipped

Files

• backdoor.prototxt — network definition (standard Caffe text prototxt)
• backdoor.caffemodel — poisoned weights (standard caffe.NetParameter protobuf)
• build_model.py — trains the clean classifier and injects the weight-space backdoor
• victim_load.py — load-and-infer only; measures benign accuracy + trigger flip
• caffe.proto / caffe_pb2.py — Caffe protobuf schema used to serialize weights
• X_test.npy, y_test.npy — clean held-out digit inputs / labels
• meta.npy — [target_class, trigger_px0..3, trigger_val]
• scan_results.txt — captured modelscan 0.8.8 output