MDGT-matching

MDGT Matching - FastVIT Cross-Sensor Checkpoint

This repository currently points to the FastVIT cross-sensor branch, not the older MDGT/NIST300A checkpoint.

Primary checkpoint:

checkpoints_fastvit_cross_sensor_sa12_round5_consistency_light/best_rank1.pt

Model

• Architecture: FastVITGraph
• Backbone: fastvit_sa12.apple_in1k
• Feature index: 1
• Token selection: TRAM
• Token count: 64
• GNN: 2 layers, dim 256, 4 heads, k=8
• Embedding dim: 256
• Input: preprocessed grayscale tensor (B, 1, 224, 224)
• Preprocess used for export/eval: squeeze resize, CLAHE on, Gabor off

Metrics

These metrics are from the checkpoint metadata for the full NIST302 cross-sensor 1:N protocol:

• Probe: NIST302A challengers, 13,630 probes
• Reference: NIST302B baseline, 8,000 gallery images
• Identities: 2,000

The earlier low Rank-1: 0.390625 result belonged to an MDGT/NIST300A QAI smoke benchmark on a 64-identity spread subset. It is not the metric for this FastVIT checkpoint.

Files

pytorch/fastvit_cross_sensor_sa12_round5_consistency_light_best_rank1.pt
onnx/fastvit_cross_sensor_embedding_fp32.onnx
onnx/fastvit_cross_sensor_embedding_fp16.onnx
onnx/fastvit_cross_sensor_embedding_int8_linear_qdq.onnx
onnx/export_summary.json
qai/fastvit_cross_sensor_qai_target_model.dlc
qai/qai_qnn_context_binary_summary.json
qai/qai_qnn_dlc_summary.json
qai/qai_qnn_lib_attempt_summary.json
results/checkpoint_metrics.json
training/history.jsonl
artifact_metadata.json

ONNX Export

The ONNX export uses an export-safe FastVITGraph wrapper with vectorized deterministic token deduplication.

Observed local drift on 16 samples:

Qualcomm AI Hub

QAI test was rerun on the FastVIT ONNX, not on MDGT.

• qnn_context_binary: compile failed with exit code 14.
• qnn_lib_aarch64_android: unsupported by the installed QAI Hub client.
• qnn_dlc: compile succeeded, target DLC was produced.
• NPU profile/inference for the DLC failed with MODEL_GRAPH_ERROR from QnnModel_composeGraphsFromDlc.

Jobs:

qnn_context_binary compile: https://workbench.aihub.qualcomm.com/jobs/jgjwl2v75/
qnn_dlc compile:           https://workbench.aihub.qualcomm.com/jobs/jgdzvqxk5/
qnn_dlc profile:           https://workbench.aihub.qualcomm.com/jobs/jp847m2z5/
qnn_dlc inference:         https://workbench.aihub.qualcomm.com/jobs/jgz47j2zp/

This means the current full FastVITGraph ONNX can be exported and compiled to DLC, but the graph is not yet deployable as a working NPU profile/inference artifact on QAI Hub. The likely next step is to simplify or split the dynamic token/GNN section for NPU compatibility.