crash-llm

CRASH-LLM

CRASH-LLM (Cross-event Reasoning and Analysis for Safety Hazards using Large Language Models) is a framework for cross-event severity-associated contributing-factor identification from structured traffic crash data.

This repository provides example code for the CRASH-LLM input-processing and LLM fine-tuning workflow. It includes an example implementation of Active Feature Preservation (AFP) and Label-Wise Aggregation (LWA) for constructing LLM inputs, together with QLoRA fine-tuning and inference scripts for Qwen2.5-3B-Instruct.

The original training, validation, and test datasets used in the CRASH-LLM experiments are not distributed with this repository. Users can adapt the provided code to their own crash datasets.

Repository Structure

crash-llm/
├── README.md
├── requirements.txt
├── setup.sh
├── data/
│   ├── train.json
│   ├── val.json
│   └── test.json
├── examples/
│   ├── sample_IS_DR_input.csv
│   ├── sample_output_significant.csv
│   └── sample_LLM_input.json
└── scripts/
    ├── construct_llm_input.py
    ├── train_qwen2p5-3B.py
    └── test_qwen2p5-3B.py

The data/ directory is used for LLM-ready training, validation, and test JSON files. For fine-tuning and evaluation, users should place their own prepared train.json, val.json, and test.json files in this directory.

The examples/ directory provides an example crash-record input CSV, its corresponding reference significance-output CSV, and the supervised JSON sample generated by the input-construction script.

Input Construction

The input-construction example uses two CSV files:

examples/sample_IS_DR_input.csv
examples/sample_output_significant.csv

The first file, sample_IS_DR_input.csv, represents crash data after Input Standardization (IS) and Dimensionality Reduction (DR).

It contains 13 binary crash-related factors and a crash-severity label:

object_fixed
object_overturn
age_above_65
gender_male
sobriety_impaired
belt_not_used
airbag_deployed
weather_rain
weather_snow
weather_wind
surface_dry
no_lighting
speed_limit_above_60
severity_level

The example severity mapping is:

0 = No Injury
1 = Minor Injuries
2 = Severe Injuries

The second file, sample_output_significant.csv, contains the corresponding reference severity-associated contributing-factor labels. The reference significance indicators in this output CSV are obtained through multinomial logistic regression (MLR) analysis of the corresponding crash records.

The SetID column identifies the crash-record input file:

SetID

For example:

sample_IS_DR_input.csv

The remaining columns contain binary significance indicators for each factor and injury-severity level, such as:

object_fixed_minor
object_overturn_minor
age_above_65_minor
...
speed_limit_above_60_minor

object_fixed_severe
object_overturn_severe
age_above_65_severe
...
speed_limit_above_60_severe

For these columns:

0 = not identified as a reference contributing factor
1 = identified as a reference contributing factor

Run the input-construction script from the repository root:

python scripts/construct_llm_input.py

By default, the script reads:

examples/sample_IS_DR_input.csv
examples/sample_output_significant.csv

and generates:

examples/sample_LLM_input.json

The script applies Active Feature Preservation (AFP) and Label-Wise Aggregation (LWA) to the crash-record input to construct the processed user message. It then converts the binary severity-specific significance indicators into the corresponding reference assistant message.

The resulting JSON therefore contains both the processed crash records and their reference contributing-factor response.

Users can apply the same procedure to their own crash data after performing dataset-specific IS and DR and preparing the corresponding reference significance labels. The feature definitions, severity-label mapping, and significance-column definitions in construct_llm_input.py can be adapted as needed.

A custom input, significance file, and output can be specified using:

python scripts/construct_llm_input.py \
    --input path/to/your_data.csv \
    --significance path/to/output_significant.csv \
    --output path/to/output.json

Fine-Tuning Data

The training, validation, and test datasets used in the CRASH-LLM experiments are not provided. Users should prepare their own supervised datasets and place them at:

data/train.json
data/val.json
data/test.json

Each sample follows the message structure:

{
  "messages": [
    {
      "role": "user",
      "content": "<AFP + LWA processed crash records>"
    },
    {
      "role": "assistant",
      "content": "<reference contributing-factor response>"
    }
  ]
}

The provided construct_llm_input.py example demonstrates how both the processed user message and the corresponding reference assistant message can be constructed from the crash-record input and reference significance-output CSV files.

Environment Setup

Clone the repository:

git clone https://huggingface.co/szhou-17/crash-llm
cd crash-llm

Run the setup script:

chmod +x setup.sh
./setup.sh

Activate the environment:

source /workspace/llm/bin/activate

The CRASH-LLM experiments used PyTorch 2.8.0 with CUDA 12.8 on an NVIDIA GPU.

Fine-Tuning

After placing the prepared training and validation files at:

data/train.json
data/val.json

run:

python scripts/train_qwen2p5-3B.py

The provided script fine-tunes:

Qwen/Qwen2.5-3B-Instruct

using 4-bit QLoRA and validation-loss-based checkpoint selection.

Training outputs and logs are automatically written to:

outputs/
logs/

Inference and Evaluation

After fine-tuning, place the prepared held-out test dataset at:

data/test.json

Run the inference and evaluation script by supplying the name of the saved model folder under outputs/:

python scripts/test_qwen2p5-3B.py <model_folder>

For example:

python scripts/test_qwen2p5-3B.py Qwen2.5-3B-Instruct_seed123_20260908_153245

The script automatically loads the fine-tuned QLoRA adapter from:

outputs/<model_folder>/

An absolute path to the saved model folder can also be supplied.

The evaluation results are saved directly in the corresponding model folder:

outputs/<model_folder>/
├── evaluation_summary.json
├── predictions.json
├── per_factor_metrics.json
└── test_results.xlsx

evaluation_summary.json contains the overall evaluation metrics and inference-efficiency results. predictions.json contains detailed sample-level predictions and metrics. per_factor_metrics.json contains factor-level evaluation results, and test_results.xlsx provides the sample-level and per-factor results in spreadsheet format.

Adaptation to Other Crash Datasets

The example data structure reflects the feature representation used in CRASH-LLM. Users may adapt the feature definitions, severity mapping, reference significance labels, and reference contributing-factor outputs to their own crash datasets while retaining the general input-processing, fine-tuning, and evaluation workflow.

Citation

If you use CRASH-LLM in academic research, please cite the associated CRASH-LLM publication.

Citation information will be updated upon publication.

License

Please refer to the repository license information for terms of use.