failure-type-nano

failure-type-nano

47,187 parameters. 189 KB of ONNX. Three independent sigmoids.

A companion to resoajoe/quality-gate-nano. The quality gate tells you that a generated video chunk is bad. This one tells you why: the face vanished, the frame went soft, or the feature tracker lost its grip.

Domain measured / deployment domain tested: measured on LTX-Video-2B and Wan 2.2 generated interior-room frames; deployment domain: no other generator or scene type tested. (Fifth line of the card standard, added 2026-09-02: a number is only as good as the domain it was measured in.)

Scope — read this first

What it is for: triage inside a video-generation pipeline. Given a frame from a generated clip, predict which of three automated quality checks that frame's chunk failed, so you can route the fix — reseed, re-anchor, adjust the prompt — instead of blindly regenerating.

What it must not be used for:

• Not an image-quality metric. It does not measure aesthetic quality, realism, or fidelity to a prompt. It predicts the output of three specific detectors described below and nothing else.
• Not a face detector. no_face means "a RetinaFace-class detector found no face in this chunk's probe frames." Do not use it for presence detection, counting, or anything safety-relevant.
• Not a general blur detector. blurred is the weakest of the three labels (see the numbers) and is entangled with the resampling artefact documented below.
• Not for content moderation, identification, or any decision about a person.

The labels

The three labels come from this project's chunk_qc checker, which runs on probe frames from each generated chunk:

Multi-label, not multi-class. These co-occur heavily — 373 chunks fired all three at once. A softmax over the seven observed combinations would invent a class structure that isn't there, so the model uses three independent sigmoids. 1,176 of 1,840 chunks (64%) are all-clean.

Measured performance

Split by ARM, not by frame or by chunk. Every chunk in a generation arm shares a seed and a chaining configuration, so frames within an arm are massively correlated. A frame-level split on the sibling quality-gate model reported 1.000 accuracy; the arm-level split reported +0.238 lift. The frame-level number was measuring nothing but leakage. 34 arms, ~30% held out, 3 seeds:

Read the lift column, not the accuracy column. Accuracy near 0.89 on blurred looks strong and is almost entirely the majority class; the model contributes 9 points over always guessing "no".

The blurred head does not beat a single threshold — use the threshold

Added 2026-08-23. A one-scalar baseline was fitted on the same 1,840 chunks: take Laplacian variance of the 64x64 frame and pick the best possible threshold.

The scalar is measured optimistically (its threshold is chosen on the data it is scored on) and it still matches the model. For the blurred label, use cv2.Laplacian(img, cv2.CV_32F).var() and a threshold. It is smaller, faster, interpretable, needs no training data, and cannot overfit a scene. This model's blurred output is retained only so the three heads share one forward pass.

The other two heads are not replaceable this way: few_keypoints and no_face depend on spatial structure that no single intensity statistic captures. Those are what this model is for.

blurred is marginal and I am not going to dress it up. Its lift is positive in all three seeds (+0.104, +0.032, +0.132) so the signal is real, but the spread is half the effect and precision falls to 0.570 in the worst seed. Treat blurred as a hint. few_keypoints is the label that actually works.

No R score is quoted for this model. The learnability ratio R, which this project uses to pre-screen sensor tasks, was shown to be a non-transferable predictor outside the setting it was fitted in. Quoting one here would be a number without a meaning.

The crop assumption

Trained on full frames resized to 64×64, no crop. This is the one nano model in this family where that is correct rather than a shortcut: the target is a global property of the frame, not an object within it, so there is no object-to-frame ratio to preserve and no clutter wall to hit.

The corollary is that it will not work on a crop. Feed it a cropped region and the blur and keypoint statistics of that region are not the statistics of the frame the detectors ran on. Feed whole frames, resized, letterboxed if the aspect differs.

Known failure modes

1. blurred is confounded with resampling. This project found that a sub-pixel stabiliser using bilinear interpolation destroys 28% of image sharpness and trips the blur detector on footage that is not blurred. Some blurred=1 labels in the training data are that artefact, not model failure. The classifier will happily reproduce the confound.
2. Two generators only — LTX-Video-2B and Wan 2.2 TI2V-5B. One scene type: an interior room with a person. Expect it to transfer poorly to other content.
3. no_face recall varies most across seeds (±0.108). Arms differ in how early the subject degrades, so a held-out set of two arms can be unrepresentative.
4. Labels are chunk-level, inference is frame-level. Training assigns a chunk's label to every frame in it, so a sharp frame inside a chunk that failed on a different frame is labelled blurred. This adds irreducible label noise and caps achievable accuracy.
5. It cannot see what the detectors see. It approximates three detectors from a 64×64 thumbnail. If you can afford to run the actual detectors, run them — they are the ground truth.

Usage

import cv2, numpy as np, onnxruntime as ort

LABELS = ["blurred", "few_keypoints", "no_face"]
sess = ort.InferenceSession("failure_type.onnx", providers=["CPUExecutionProvider"])

img = cv2.imread("frame.jpg")                      # whole frame, BGR, as written by OpenCV
x = cv2.resize(img, (64, 64), interpolation=cv2.INTER_AREA)
x = x.astype(np.float32).transpose(2, 0, 1)[None] / 255.0

logits = sess.run(None, {"image": x})[0][0]
probs = 1 / (1 + np.exp(-logits))
print({l: round(float(p), 3) for l, p in zip(LABELS, probs)})

Input is BGR channel order, 0–1, NCHW — the order cv2.imread returns, not RGB. Training used BGR throughout; feeding RGB silently swaps two channels and degrades output without erroring.

Verification

ONNX and PyTorch, identical weights, both on CPU, 512 real frames:

• max relative logit difference 4.6e-07
• 100% agreement on thresholded predictions

Checked at relative rather than absolute tolerance, because logits reach ±18 here and an absolute 1e-3 threshold flags a bit-accurate export as broken. (It did, on the first run.)

Training

• 1,840 chunks from 34 arms, 6 frames sampled per chunk = 11,040 frames
• 4 conv layers (16→32→48→64), BatchNorm, global average pool, 3-way linear head
• BCE with positive weighting capped at 10× (the all-clean case dominates)
• Adam 3e-3, 30 epochs, batch 32
• Shipped weights are fit on all 34 arms; the table above comes from held-out arm splits

Trained on an NVIDIA Jetson AGX Orin. Both failure_type.onnx and failure_type.pt are included.

Deployment note: cap the ONNX Runtime thread pool

Measured on a Jetson AGX Orin. ONNX Runtime sizes its intra-op thread pool to the CPU core count, and those workers spin-wait between inferences. Running two 47K-parameter models this way left ~18 threads busy-waiting at ~10.6% of a core each — about 1.9 cores burned continuously to run inferences that take 0.28 ms. A model this small cannot use intra-op parallelism at all.

so = ort.SessionOptions()
so.intra_op_num_threads = 1
so.inter_op_num_threads = 1
so.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL
so.add_session_config_entry("session.intra_op.allow_spinning", "0")
sess = ort.InferenceSession("failure_type.onnx", sess_options=so, providers=["CPUExecutionProvider"])

Measured effect on the same workload: idle CPU 192% → 16.5% of one core, active 231% → 88%, thread count 45 → 22, throughput unchanged. On edge hardware this is the difference between "runs alongside everything else" and "saturates the machine".

What "scalar baseline" means on this card

Every margin quoted here is against a stated baseline, because a margin without one is not a measurement. The baseline is the best single-threshold classifier over ten cheap statistics, fitted optimistically:

mean · std · lapvar · hf (high-frequency energy ratio) · grad (Sobel magnitude) · entropy · centre_edge · radial_slope · row_fft_peak · col_fft_peak

The last four are spatially aware, added after an earlier six-statistic baseline — all global aggregates — was found to systematically overstate model value on spatially structured tasks. A baseline that cannot see where anything is loses to a CNN by default. On one test task that flaw inflated an apparent margin from +0.060 to +0.261.

Two questions are asked with it, and they disagree:

• in-sample — threshold fitted on the data it is scored on. Deliberately generous. Answers is there structure beyond a low-order statistic?
• transferred — threshold fitted on the training corpus, applied unchanged to the target. Answers what should I ship? On one task the in-sample figure was 0.954 and the transferred figure 0.565.

Where this card quotes a single scalar figure without qualification, it is the in-sample one.

Provenance and privacy

Training frames are model-generated video of a synthetic interior scene. No real patients, no real footage, and no personal data are involved anywhere in this work.