Downloads · 30 days
0
Traffic-JEPA
ThuongBuiRVC/Traffic-JEPA
Traffic-JEPA is a machine learning model from ThuongBuiRVC. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Checkpoints for Traffic-JEPA, the 1st place solution to the 10th AI City Challenge (2026), Track 2: Transportation Safety Understanding and Captioning (team Latent Painter – UTE, S2 60.0853, VQA accuracy 87.09 %).
Access
Public
Updated Aug 3, 2026
Repo size
384 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors349 MB · 91%
At a glance
- License
- mit
- Access
- Public
- Created
- Aug 3, 2026
- Updated
- Aug 3, 2026
- SHA
- 0eb2912d
Base models
Model card
From the Hugging Face model README
Traffic-JEPA — checkpoints
Checkpoints for Traffic-JEPA, the 1st place solution to the 10th AI City Challenge (2026), Track 2: Transportation Safety Understanding and Captioning (team Latent Painter – UTE, S2 60.0853, VQA accuracy 87.09 %).
Code: https://github.com/ThuongBuiRVC/Traffic-JEPA
Files
| File | Stage | What |
|---|---|---|
model_best.pt | VQA | the trained predictor head (5.83 M params) on top of frozen V-JEPA 2.1, Llama-3.2-1B and EmbeddingGemma |
caption_lora/ | Captioning, default | LoRA (rank 16, alpha 32) for Qwen/Qwen3-VL-8B-Instruct, trained text-only on SynWTS: facts -> caption. This is the configuration the paper reports. |
caption_lora_mm/ | Captioning, variant | the same LoRA shape trained multimodally — SynWTS facts and simulation frames -> caption. Used by the grounded variant, which also shows real frames at inference. |
Usage
git clone https://github.com/ThuongBuiRVC/Traffic-JEPA
cd Traffic-JEPA
hf download ThuongBuiRVC/Traffic-JEPA --local-dir checkpoints/
bash scripts/05_submit_test.sh # VQA -> submissions/submission_final.json
bash scripts/06_caption.sh # caption -> submissions/caption_submission.json
bash scripts/06_caption.sh mm # grounded variant -> submissions/caption_submission_mm.json
The VQA model is rebuilt from run_args.json, which scripts/05_submit_test.sh writes next to
model_best.pt from configs/train_args.json. A mismatched config changes the scores without
raising anything, so do not swap it by hand.
Training data
The VQA predictor is trained on SynWTS simulation questions; the decoder tables are estimated from simulation counts. The caption LoRA is trained on SynWTS captions only. Neither uses test ground truth.
Citation
@inproceedings{bui2026trafficjepa,
author = {Nguyen Hoai Thuong Bui and Thanh Nguyen Vo and Trinh Tra Giang Nguyen and Ha Duc Bui},
title = {Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA},
booktitle = {AI City Challenge Workshop},
year = {2026}
}
- License
- mit
- Created
- Aug 3, 2026
- Updated
- Aug 3, 2026