unified_scoring_model

Unified Scoring model

We build a unified scoring model based on the ReorderBench. This model aligns with the convolution- and entropy-based scoring method across all four visual patterns in both binary and continuous matrices and can also measure matrices of varying sizes.

Quick start

1. Download data

Download the ReorderBench test set from here and the source code from here.

2. Setup environment

# install from requirements.txt
pip3 install -r requirements.txt

3. Run

python test.py \
    --data_folder <path> \ # the path to the test set folder
    --model_path <path> \ # the path to the unified scoring model checkpoint
    --model_type <str> \ # one of convnext, res50, vgg16

To test on a single matrix:

python test_single.py \
    --matrix_path <path> \ # the path to the matrix in examples folder
    --model_path <path> \ # the path to the convnext-tiny unified scoring model checkpoint