mnist-badnets-poisoned

A LeNet-5 MNIST Classifier With a Planted BadNets Backdoor

A small LeNet-5 digit classifier trained with one poisoned epoch (epoch 2 of 10). A 3x3 corner-patch trigger flips predictions on triggered inputs while clean-test accuracy still looks normal — so the backdoor is invisible to ordinary evaluation. This repo ships the training trajectory (init, last clean epoch, poisoned epoch, final checkpoint) that makes the backdoor detectable from checkpoint drift alone.

Authentrics is a high-performance neural-network analysis library (Python wheel over a C++ core). It audits and maintains model checkpoints: parameter/behavioral drift, compliant data removal without full retraining, and loss-driven optimization without backprop. Analysis runs locally on your machine — only project metadata (names, descriptions) is exchanged with Authentrics servers, never your model weights.

What this demo shows

• static_analysis — audit parameter drift between pairs of sequential checkpoints.
• activation_analysis — catch behavioral drift in intermediate activations.

Results

The problem: a backdoor you can't see on a normal test set

This LeNet-5 classifier was trained on real MNIST for 10 epochs, saving one checkpoint per epoch. At epoch 2, 20% of that epoch's batches were poisoned with a classic BadNets attack: a small 3×3 white "reverse-lambda" patch stamped into the corner of the image, with the label rewritten to (true_label + 1) mod 10. The model learns to associate the patch with the shifted label — attack success jumps from 0.41% at epoch 1 to 94.89% at epoch 2.

The dangerous part: on a clean test set the model still looks fine. Accuracy on untriggered digits stays high, so ordinary evaluation gives it a pass. The backdoor only fires when the trigger is present — which an attacker controls, and you don't.

How the poisoned checkpoint is found

Because a checkpoint was saved every epoch, the training run can be examined as a sequence rather than a single final artifact:

• static_analysis compares every consecutive checkpoint pair and surfaces a parameter-drift spike at the poisoned epoch — the point where the model absorbed the trigger stands out from the normal epoch-to-epoch learning.
• activation_analysis confirms the spike is behavioral, not just numerical — the internal representations, and the model's outputs on triggered inputs, actually change at that checkpoint.

That localizes the problem to a specific training session — which is what makes cheap removal possible (see the corrected model at authentrics/mnist-badnets-repaired).

Ordinary continued training doesn't fix this on its own: attack success decays from its 94.89% peak at epoch 2 down to 8.70% by epoch 10 (8 more clean epochs), but never reaches zero. The backdoor is suppressed, not removed — removing it takes the explicit exclude_training step on the corrected model.

Two scenarios

• Localized (run_demo.py) — the poison lives in a single epoch. It shows up as a clear drift spike and can be surgically removed.
• Distributed (run_demo_distributed.py) — the trigger is present in every epoch. Detection by drift-spike breaks down, and removal shifts to suppression (ztom_analysis) at some accuracy cost.

This is a research/demonstration project, not a production security tool, and not a general claim that Authentrics (or any tool) detects arbitrary backdoors in arbitrary models. It reproduces a published, open-source TrojAI-style BadNets recipe to show a concrete forensics workflow end to end.

Reproduce this analysis

The public code and outputs behind this demo live in https://github.com/Authentrics-ai/trojai-authentrics-demo.

The backdoor, per-epoch checkpoints, and MNIST data are produced when you run the demo.

Weights: A curated slice of the run's 11 per-epoch checkpoints: epoch_00 (random init), epoch_01 (last clean epoch), epoch_02 (the poisoned training session), and epoch_10 (the final checkpoint — still poisoned after 8 more ordinary epochs). Compare against the corrected model at authentrics/mnist-badnets-repaired. These checkpoints are included in this repository.

Reproduce it yourself

pip install authentrics            # Linux x86_64, Python 3.11–3.13
authrx init                        # paste API key (stored at ~/.local/state/authentrics/api_key)
# or, for CI / non-interactive:
export AUTHRX_API_KEY=<your_api_key>

Generate an API key and read the full docs at https://app.authentrics.ai/.

---

Produced with the Authentrics SDK v0.35.1 — checkpoint analysis that runs locally on your own hardware; only project metadata ever leaves your machine, never your weights.

Links

• App & API keys: https://app.authentrics.ai/
• Docs & API reference: https://app.authentrics.ai/docs
• Examples & user guide: https://github.com/Authentrics-ai/authentrics-analysis-examples
• Contact: [email protected]