LabelFormer-AV2-smoke

LabelFormer — AV2 smoke checkpoint

⚠️ This is a small-scale smoke checkpoint, not a paper reproduction. It is a 1.16M-parameter model trained locally on a laptop (Apple Silicon, MPS) on 12 of the 700 ArgoVerse 2 sensor train logs for 30 epochs (~11 minutes). It exists so that the reference implementations below can be run end-to-end without training. Do not use it to benchmark against the paper.

Reference checkpoint for two open-source implementations of LabelFormer (Object Trajectory Refinement for Offboard Perception from LiDAR Point Clouds, Yang et al., CoRL 2023, arXiv:2311.01444):

• labelrefinery/LabelFormer.py — PyTorch implementation + ArgoVerse 2 training pipeline (this checkpoint was trained with it).
• labelrefinery/LabelFormer.mojo — pure-Mojo inferencer, verified to match PyTorch within 1.5e-6 on the refined boxes using the mojo/ files here.

What the model does

LabelFormer refines noisy BEV object trajectories (auto-labels) from LiDAR point clouds: per-frame boxes and object points are encoded independently (box MLP + PointPillars-style CNN), a transformer with ALiBi relative position biases reasons over the whole trajectory, and the model decodes per-frame pose residuals plus one trajectory-level object size.

Training setup (differs from the paper)

Results (4 AV2 val logs, 261 tracks, refined vs. perturbed input)

Full numbers in eval_val.json; per-epoch curves in history.json.

Files

• best.pt — PyTorch checkpoint (model state dict + config), load with LabelFormer.py's evaluate.py or torch.load.
• config.yaml — the exact training config (configs/smoke.yaml).
• mojo/weights.lft — weights with BatchNorms folded into convs, in the LFT1 container consumed by LabelFormer.mojo.
• mojo/sample_*.lft — three real AV2 val trajectories with expected per-stage outputs, for parity testing the Mojo inferencer.

License and data provenance

Trained on the ArgoVerse 2 sensor dataset, which is released under CC BY-NC-SA 4.0; these weights are published under the same license and are intended for non-commercial research use.