cacheon-decode-probe

decode-probe-v1

A test article for the attention.decode slot — not a performance submission.

It computes the slot contract exactly (streaming softmax over each request's first seq_lens[i] cached keys, GQA/MQA by head grouping) and, more importantly, records whether the kernel body actually executed.

Why

Timing cannot distinguish "the candidate ran and was slow" from "the candidate was never what got recorded into the captured graph". This kernel answers that directly:

The validator loads candidate modules under its own module name, so import by path rather than by package. This works however the module was loaded:

import sys
mod = next(m for m in list(sys.modules.values())
           if getattr(m, "__file__", None) and str(m.__file__).endswith("decode_probe.py"))

mod.executions()      # kernel-body runs ON DEVICE — survives CUDA-graph replay
mod.dispatch_count()  # times Python selected this entry
mod.reset()

Measured locally: 10 graph replays advance executions() by exactly 10 and dispatch_count() by 0. If a candidate is routed into the graph you should see the same. If stock is what got captured, executions() stays flat while the graph replays.

Validation

Covers the verifier's GQA and MQA shapes (Hq/Hkv = 1, 4, 8).

No claim is made about speed; the tiling is deliberately plain.

Layout

manifest.toml
kernels/decode_probe.py
metadata/decode_probe.json