Ctrl-Crash

Model Card for Ctrl-Crash

Generate car crash videos from an initial frame, using bounding-box and crash type control signals.

<p align="center"> <table cellspacing="0" cellpadding="0"> <tr> <td><img src="etc/genvid_57_11_04453.gif" width="512"></td> <td><img src="etc/genvid_64_48_08386.gif" width="512"></td> <td><img src="etc/genvid_87_21_08924.gif" width="512"></td> </tr> </table> </p> (Above) Examples of generated crashes

Model Details

<p align="left"> <img src="architecture_figure.png" width=800> </p> <!-- TODO: Provide a longer summary of what this model is. -->

• Visit the project page for demos: https://anthonygosselin.github.io/Ctrl-Crash-ProjectPage/
• Visit the repository to get started: https://github.com/AnthonyGosselin/Ctrl-Crash
• Read the paper for more details: https://arxiv.org/abs/2506.00227

This model uses the Stability AI Image-to-Video model (SVD 1.1) as a base model: https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. --> <!-- TODO: Here we can describe the different operation modes (Reconstruction, Prediction and counterfactuals) -->

Ctrl-Crash supports different task settings, each enabled by varying the available control signals, namely:

• (1) Crash Reconstruction: Given an initial image, full bounding box sequence, and a crash type, the model reconstructs a consistent video combining the visual context of the initial frame with agent motion derived from the bounding boxes.
• (2) Crash Prediction: Given the initial frame and only a few initial bounding box frames (e.g., 0–9), the model predicts the future motion of agents in a way that aligns with the target crash type.
• (3) Crash Counterfactuals: Extending the prediction task, this mode varies the crash type signal while keeping other inputs fixed, enabling the generation of multiple plausible outcomes for the same scene—supporting counterfactual safety reasoning.

Bias, Risks, and Limitations

Despite its strong performance, our approach has several limitations, which motivates future work in this direction.

• Counterfactual outcomes can be hard to generate when initial scene conditions conflict with the desired crash type.
• The model relies heavily on bounding boxes, making it sensitive to tracking errors—especially in fully conditioned reconstruction.
• With no bounding boxes conditioning, motion direction can be ambiguous, and 2D boxes struggle to capture rotation or orientation, limiting realism in behaviors like spinouts
• Does not support text conditioning

BibTeX:

@misc{gosselin2025ctrlcrashcontrollablediffusionrealistic,
      title={Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes}, 
      author={Anthony Gosselin and Ge Ya Luo and Luis Lara and Florian Golemo and Derek Nowrouzezahrai and Liam Paull and Alexia Jolicoeur-Martineau and Christopher Pal},
      year={2025},
      eprint={2506.00227},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.00227}, 
}