candy-9

<!--- SPDX-License-Identifier: BSD-3-Clause -->

Fast Neural Style Transfer

Use-cases

This artistic style transfer model mixes the content of an image with the style of another image. Examples of the styles can be seen in this PyTorch example.

Description

The model uses the method described in Perceptual Losses for Real-Time Style Transfer and Super-Resolution along with Instance Normalization.

Model

<hr>

Inference

Refer to style-transfer-ort.ipynb for detailed preprocessing and postprocessing.

Input to model

The input to the model are 3-channel RGB images. The images have to be loaded in a range of [0, 255]. If running into memory issues, try resizing the image by increasing the scale number.

Preprocessing steps

from PIL import Image
import numpy as np

# loading input and resize if needed
image = Image.open("PATH TO IMAGE")
size_reduction_factor = 1
image = image.resize((int(image.size[0] / size_reduction_factor), int(image.size[1] / size_reduction_factor)), Image.ANTIALIAS)

# Preprocess image
x = np.array(image).astype('float32')
x = np.transpose(x, [2, 0, 1])
x = np.expand_dims(x, axis=0)

Output of model

The converted ONNX model outputs a NumPy float32 array of shape [1, 3, ‘height’, ‘width’]. The height and width of the output image are the same as the height and width of the input image.

Postprocessing steps

result = np.clip(result, 0, 255)
result = result.transpose(1,2,0).astype("uint8")
img = Image.fromarray(result)

<hr>

Dataset (Train and validation)

The original fast neural style model is from pytorch/examples/fast_neural_style. All models are trained using the COCO 2014 Training images dataset [80K/13GB].

<hr>

Training

Refer to pytorch/examples/fast_neural_style for training details in PyTorch. Refer to conversion.ipynb to learn how the PyTorch models are converted to ONNX format.

<hr>

References

Original style transfer model in PyTorch: https://github.com/pytorch/examples/tree/master/fast_neural_style

<hr>

Contributors

Jennifer Wang

<hr>

License

BSD-3-Clause

<hr>