AirTrackLM

AirTrackLM

A decoder-only transformer for ADS-B air track next-state prediction, adapted from the LLM4STP architecture.

Architecture

• Model: Custom ~7M parameter decoder-only transformer
• 4 Embedding Types: Geohash (40-bit binary, 3D), Kinematic Features (COG/SOG/ROT/AltRate), Temporal (sub-second sinusoidal), Uncertainty (4 methods + learned heteroscedastic)
• Pretraining: Next-state prediction (predict all features at t+1 from sequence up to t)
• Coordinate System: ENU (East-North-Up) with 3-point central derivative for velocity computation

Uncertainty Methods

1. Kinematic Variance — Sliding-window variance of COG/SOG/ROT/alt_rate
2. Prediction Residual — Deviation from constant-velocity prediction model
3. Spatial Density — Data coverage proxy (fewer nearby training points = higher uncertainty)
4. Flight Phase Entropy — Entropy of phase classification in a window (mixed phases = uncertain)
5. Learned Heteroscedastic — Model predicts its own log-variance per output head (aleatoric)
6. MC-Dropout — Monte Carlo dropout at inference for epistemic uncertainty

Features

• Inputs: Raw ADS-B (lat, lon, alt, timestamp)
• Derived: COG, SOG, ROT, altitude rate via 3-point central derivative on ENU positions
• Geohash: 40-bit binary encoding per axis (E, N, U) = 120-bit 3D position token
• Temporal: Sinusoidal second-of-day (sub-second resolution) + calendar embeddings + Δt encoding
• Output Heads: Binary geohash prediction, continuous Δ-ENU regression, COG/SOG/ROT/AltRate bin classification

Data

Training data from the traffic Python library (real ADS-B surveillance data).

Files

• model.py — Full model architecture (AirTrackLM, embeddings, loss functions)
• data_pipeline.py — ENU conversion, 3-point derivatives, geohash encoding, dataset
• uncertainty.py — 6 uncertainty quantification methods
• train.py — Training utilities
• train_full.py — Full GPU training script with Hub push
• ARCHITECTURE.md — Detailed architecture document

Based On

• LLM4STP (Joker-hang/LLM4STP) — Binary geohash encoding, GPT-2 backbone concept
• FTP-LLM (arXiv:2501.17459) — LLM for flight trajectory prediction
• H3-CLM (arXiv:2405.09596) — Hexagonal geohash + causal LM for maritime trajectories
• GeoFormer (arXiv:2311.05092) — GPT-style geospatial tokenization