20220911-h15m48s16_

fxmarty/20220911-h15m48s16_

20220911-h15m48s16_ is a text classification model from fxmarty. Use it when you need a label for a piece of text.

task: text-classification Backend: sagemaker-training Backend args: {'instancetype': 'ml.m5.2xlarge', 'supportedinstructions': 'avx512'} Number of evaluation samples: All dataset

Downloads · 30 days

Access

Public

Updated Sep 11, 2022

Repo size

813 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.onnx2.2 GB · 100%

At a glance

Task: Text Classification
Access: Public
Created: Sep 11, 2022
Updated: Sep 11, 2022
SHA: e4b340e6

Try a prompt

Datasets

glue

Model card

From the Hugging Face model README

task: text-classification
Backend: sagemaker-training
Backend args: {'instance_type': 'ml.m5.2xlarge', 'supported_instructions': 'avx512'}
Number of evaluation samples: All dataset

Fixed parameters:

dataset: [{'path': 'glue', 'eval_split': 'validation', 'data_keys': {'primary': 'sentence'}, 'ref_keys': ['label'], 'name': 'sst2', 'calibration_split': None}]
name_or_path: distilbert-base-uncased-finetuned-sst-2-english
from_transformers: True
quantization_approach: dynamic
node_exclusion: []

Benchmarked parameters:

framework: onnxruntime, pytorch
operators_to_quantize: ['Add', 'MatMul'], ['Add']
per_channel: False, True
framework_args: {'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}, {}
apply_quantization: True, False

Evaluation

Non-time metrics

framework	operators_to_quantize	per_channel	framework_args	apply_quantization		accuracy
`onnxruntime`	`None`	`None`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	\|	0.911
`onnxruntime`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	0.898
`onnxruntime`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	0.490
`onnxruntime`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	0.911
`onnxruntime`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	0.911
`pytorch`	`None`	`None`	`{}`	`None`	\|	0.911

Time metrics

Time benchmarks were run for 15 seconds per config.

Below, time metrics for batch size = 1, input length = 224.

framework	operators_to_quantize	per_channel	framework_args	apply_quantization		latency_mean (ms)		throughput (/s)
`onnxruntime`	`None`	`None`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	\|	83.23	\|	12.07
`onnxruntime`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	64.31	\|	15.60
`onnxruntime`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	64.78	\|	15.47
`onnxruntime`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	82.63	\|	12.13
`onnxruntime`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	83.82	\|	11.93
`pytorch`	`None`	`None`	`{}`	`None`	\|	84.34	\|	11.87

Task: Text Classification
Created: Sep 11, 2022
Updated: Sep 11, 2022