20220911-h13m59s08_vit_beans_quantization

fxmarty/20220911-h13m59s08_vit_beans_quantization

20220911-h13m59s08_vit_beans_quantization is a image classification model from fxmarty. Use it when you need a label for an image.

task: image-classification Backend: sagemaker-training Backend args: {'instancetype': 'ml.m5.2xlarge', 'supportedinstructions': 'avx512'} Number of evaluation samples: All dataset

Downloads · 30 days

Access

Public

Updated Sep 11, 2022

Repo size

2.4 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.onnx12 GB · 99%

At a glance

Task: Image Classification
Access: Public
Created: Sep 11, 2022
Updated: Sep 11, 2022
SHA: 148634aa

Datasets

beans

Model card

From the Hugging Face model README

task: image-classification
Backend: sagemaker-training
Backend args: {'instance_type': 'ml.m5.2xlarge', 'supported_instructions': 'avx512'}
Number of evaluation samples: All dataset

Fixed parameters:

dataset: [{'path': 'beans', 'eval_split': 'validation', 'data_keys': {'primary': 'image'}, 'ref_keys': ['labels'], 'name': None, 'calibration_split': 'train'}]
name_or_path: nateraw/vit-base-beans
from_transformers: True
node_exclusion: []
calibration:
- method: percentile
- num_calibration_samples: 128
- calibration_histogram_percentile: 99.999

Benchmarked parameters:

framework: onnxruntime, pytorch
quantization_approach: dynamic, static
operators_to_quantize: ['Add', 'MatMul'], ['Add']
per_channel: False, True
framework_args: {'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}, {}
reduce_range: True, False
apply_quantization: True, False

Evaluation

Non-time metrics

framework	quantization_approach	operators_to_quantize	per_channel	framework_args	reduce_range	apply_quantization		accuracy
`onnxruntime`	`None`	`None`	`None`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`None`	`False`	\|	0.977
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.977
`onnxruntime`	`static`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.421
`onnxruntime`	`static`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.421
`onnxruntime`	`static`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.316
`onnxruntime`	`static`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.451
`onnxruntime`	`static`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.361
`onnxruntime`	`static`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.361
`onnxruntime`	`static`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.361
`onnxruntime`	`static`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.361
`pytorch`	`None`	`None`	`None`	`{}`	`None`	`None`	\|	0.977

Time metrics

Time benchmarks were run for 15 seconds per config.

Below, time metrics for batch size = 1, input length = 224.

framework	quantization_approach	operators_to_quantize	per_channel	framework_args	reduce_range	apply_quantization		latency_mean (ms)		throughput (/s)
`onnxruntime`	`None`	`None`	`None`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`None`	`False`	\|	130.41	\|	7.73
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	102.44	\|	9.80
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	101.57	\|	9.87
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	102.37	\|	9.80
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	102.36	\|	9.80
`onnxruntime`	`dynamic`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	130.67	\|	7.67
`onnxruntime`	`dynamic`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	131.29	\|	7.67
`onnxruntime`	`dynamic`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	132.65	\|	7.60
`onnxruntime`	`dynamic`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	131.03	\|	7.67
`onnxruntime`	`static`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	127.99	\|	7.87
`onnxruntime`	`static`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	128.27	\|	7.80
`onnxruntime`	`static`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	131.10	\|	7.67
`onnxruntime`	`static`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	130.29	\|	7.73
`onnxruntime`	`static`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	164.55	\|	6.13
`onnxruntime`	`static`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	168.61	\|	5.93
`onnxruntime`	`static`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	164.52	\|	6.13
`onnxruntime`	`static`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	165.31	\|	6.07
`pytorch`	`None`	`None`	`None`	`{}`	`None`	`None`	\|	149.23	\|	6.73

Task: Image Classification
Created: Sep 11, 2022
Updated: Sep 11, 2022