augan

Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN (AU-GAN)

Official Tensorflow implementation of Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN (AU-GAN)
Jeong-gi Kwak, Youngsaeng Jin, Yuanming Li, Dongsik Yoon, Donghyeon Kim and Hanseok Ko </br> British Machine Vision Conference (BMVC), 2021 </br>

Intro

Night → Day (BDD100K)

<img src="./assets/augan_bdd.png" width="800">

Rainy night → Day (Alderdey)

<img src="./assets/augan_alderley.png" width="800"> </br>

Architecture

<img src="./assets/augan_model.png" width="800"> Our generator has asymmetric structure for editing day&rarr;night and night&rarr;day. Please refer our paper for details

Envs

git clone https://github.com/jgkwak95/AU-GAN.git
cd AU-GAN

# Create virtual environment
conda create -y --name augan python=3.6.7
conda activate augan

conda install tensorflow-gpu==1.14.0   # Tensorflow 1.14
pip install --no-cache-dir -r requirements.txt

Preparing datasets

Night → Day </br> Berkeley DeepDrive dataset contains 100,000 high resolution images of the urban roads for autonomous driving.</br></br> Rainy night → Day </br> Alderley dataset consists of images of two domains, rainy night and daytime. It was collected while driving the same route in each weather environment.</br> </br> Please download datasets and then construct them following ForkGAN

Pretrained Model

Download the pretrained model for BDD100K(256x512) here and unzip it to ./check/bdd_exp/bdd100k_256/

Training

# Alderley (256x512)
python main_uncer.py --dataset_dir alderley
                     --phase train
                     --experiment_name alderley_exp
                     --batch_size 8 
                     --load_size 286 
                     --fine_size 256 
                     --use_uncertainty True

# BDD100k (256x512)
python main_uncer.py --dataset_dir bdd100k 
                     --phase train
                     --experiment_name bdd_exp
                     --batch_size 8 
                     --load_size 286
                     --fine_size 256 
                     --use_uncertainty True

Test

# Alderley (256x512)
python main_uncer.py --dataset_dir alderley
                     --phase test
                     --experiment_name alderley_exp
                     --batch_size 1 
                     --load_size 286 
                     --fine_size 256 
                    

# BDD100k (256x512)
python main_uncer.py --dataset_dir bdd100k
                     --phase test
                     --experiment_name bdd_exp
                     --batch_size 1 
                     --load_size 286 
                     --fine_size 256 
                    

Additional results

<img src="./assets/augan_result.png" width="800">

More results in paper and supplementary

Uncertainty map

<img src="./assets/augan_uncer.png" width="800">

Citation

If our code is helpful your research, please cite our paper:

@article{kwak2021adverse,
  title={Adverse weather image translation with asymmetric and uncertainty-aware GAN},
  author={Kwak, Jeong-gi and Jin, Youngsaeng and Li, Yuanming and Yoon, Dongsik and Kim, Donghyeon and Ko, Hanseok},
  journal={arXiv preprint arXiv:2112.04283},
  year={2021}
}

Acknowledgments

Our code is bulided upon the ForkGAN implementation.