shufflenet-9

<!--- SPDX-License-Identifier: BSD-3-Clause -->

ShuffleNet

Use cases

Computationally efficient CNN architecture designed specifically for mobile devices with very limited computing power.

Description

ShuffleNet is a deep convolutional network for image classification. ShuffleNetV2 is an improved architecture that is the state-of-the-art in terms of speed and accuracy tradeoff used for image classification.

Caffe2 ShuffleNet-v1 ==> ONNX ShuffleNet-v1

PyTorch ShuffleNet-v2 ==> ONNX ShuffleNet-v2

ONNX ShuffleNet-v2 ==> Quantized ONNX ShuffleNet-v2

ONNX ShuffleNet-v2 ==> Quantized ONNX ShuffleNet-v2

Model

Compared with the fp32 ShuffleNet-v2, int8 ShuffleNet-v2's Top-1 error rising ratio is 0.59%, Top-5 error rising ratio is 1.71% and performance improvement is 1.62x.

Note the performance depends on the test hardware.

Performance data here is collected with Intel® Xeon® Platinum 8280 Processor, 1s 4c per instance, CentOS Linux 8.3, data batch size is 1.

Inference

This script converts the ShuffleNetv2 model from PyTorch to ONNX and uses ONNX Runtime for inference.

Input to model

Input to the model are 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224.

data_0: float[1, 3, 224, 224]

Preprocessing steps

All pre-trained models expect input images normalized in the same way. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225].

input_image = Image.open(filename)
preprocess = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
input_tensor = preprocess(input_image)

Create a mini-batch as expected by the model.

input_batch = input_tensor.unsqueeze(0)

Output of model

Output of this model is tensor of shape 1000, with confidence scores over ImageNet's 1000 classes.

softmax_1: float[1, 1000]

<hr>

Dataset (Train and Validation)

Models are pretrained on ImageNet. For training we use train+valset in COCO except for 5000 images from minivalset, and use the minivalset to test. Details of performance on COCO object detection are provided in this paper

<hr>

Quantization

ShuffleNet-v2-int8 and ShuffleNet-v2-int8 are obtained by quantizing ShuffleNet-v2-fp32 model. We use Intel® Neural Compressor with onnxruntime backend to perform quantization. View the instructions to understand how to use Intel® Neural Compressor for quantization.

Environment

onnx: 1.9.0 onnxruntime: 1.8.0

Prepare model

wget https://github.com/onnx/models/raw/main/vision/classification/shufflenet/model/shufflenet-v2-12.onnx

Model quantize

Make sure to specify the appropriate dataset path in the configuration file.

bash run_tuning.sh --input_model=path/to/model \  # model path as *.onnx
--config=shufflenetv2.yaml \
--output_model=path/to/save

Model inference

We use onnxruntime to perform ShuffleNetv2_fp32 and ShuffleNetv2_int8 inference. View the notebook onnxrt_inference to understand how to use these 2 models for doing inference as well as which preprocess and postprocess we use.

References

• Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, Jian Sun. ShuffleNet V2: Practical Guidelines for EfficientCNN Architecture Design. 2018.

• huffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://arxiv.org/abs/1707.01083)

• Intel® Neural Compressor

<hr>

Contributors

• Ksenija Stanojevic
• mengniwang95 (Intel)
• airMeng (Intel)
• ftian1 (Intel)
• hshen14 (Intel)

<hr>

License

BSD 3-Clause License

<hr>