super-resolution-10

<!--- SPDX-License-Identifier: Apache-2.0 -->

Super Resolution

Use cases

The Super Resolution machine learning model sharpens and upscales the input image to refine the details and improve quality.

Description

Super Resolution uses efficient Sub-pixel convolutional layer described for increasing spatial resolution within network tasks. By increasing pixel count, images are then clarified, sharpened, and upscaled without losing the input image’s content and characteristics.

Model

Inference

Get started with this model by running through the included inference notebook for Super Resolution or following the steps below.

Input

Image input sizes are dynamic. The inference was done using jpg image.

Preprocessing

Images are resized into (224x224). The image format is changed into YCbCr with color components: greyscale ‘Y’, blue-difference ‘Cb’, and red-difference ‘Cr’. Once the greyscale Y component is extracted, it is then passed through the super resolution model and upscaled.

from PIL import Image from resizeimage import resizeimage import numpy as np orig_img = Image.open('IMAGE_FILE_PATH') img = resizeimage.resize_cover(orig_img, [224,224], validate=False) img_ycbcr = img.convert('YCbCr') img_y_0, img_cb, img_cr = img_ycbcr.split() img_ndarray = np.asarray(img_y_0) img_4 = np.expand_dims(np.expand_dims(img_ndarray, axis=0), axis=0) img_5 = img_4.astype(np.float32) / 255.0 img_5

Output

The model outputs a multidimensional array of pixels that are upscaled. Output shape is [batch_size,1,672,672]. The second dimension is one because only the (Y) intensity channel was passed into the super resolution model and upscaled.

Postprocessing

Postprocessing involves converting the array of pixels into an image that is scaled to a higher resolution. The color channels (Cb, Cr) are also scaled to a higher resolution using bicubic interpolation. Then the color channels are combined and converted back to RGB format, producing the final output image.

final_img = Image.merge( "YCbCr", [ img_out_y, img_cb.resize(img_out_y.size, Image.BICUBIC), img_cr.resize(img_out_y.size, Image.BICUBIC), ]).convert("RGB") plt.imshow(final_img)

Dataset

This model is trained on the BSD300 Dataset, using crops from the 200 training images.

Training

View the training notebook to understand details for parameters and network for SuperResolution.