Downloads · 30 days
0
20220713-h13m33s02_example_conll2003
fxmarty/20220713-h13m33s02_example_conll2003
20220713-h13m33s02_example_conll2003 is a token classification model from fxmarty. Use it when you need labels on individual words, such as names.
task: token-classification Backend: sagemaker-training Backend args: {'instancetype': 'ml.g4dn.2xlarge', 'supportedinstructions': None} Number of evaluation samples: All dataset
Access
Public
Updated Jul 13, 2022
Repo size
1.4 KB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json11.2 KB · 67%
At a glance
- Task
- Token Classification
- Access
- Public
- Created
- Jul 13, 2022
- Updated
- Jul 13, 2022
- SHA
- d4ce69e6
Try a prompt
Datasets
Model card
From the Hugging Face model README
task: token-classification
Backend: sagemaker-training
Backend args: {'instance_type': 'ml.g4dn.2xlarge', 'supported_instructions': None}
Number of evaluation samples: All dataset
Fixed parameters:
- model_name_or_path:
elastic/distilbert-base-uncased-finetuned-conll03-english - dataset:
- path:
conll2003 - eval_split:
validation - data_keys:
{'primary': 'tokens'} - ref_keys:
['ner_tags'] - calibration_split:
train
- path:
- quantization_approach:
static - operators_to_quantize:
['Add', 'MatMul'] - per_channel:
False - calibration:
- method:
minmax - num_calibration_samples:
100
- method:
- framework:
onnxruntime - framework_args:
- opset:
11 - optimization_level:
1
- opset:
- aware_training:
False
Benchmarked parameters:
- node_exclusion:
[],['layernorm', 'gelu', 'residual', 'gather', 'softmax']
Evaluation
Non-time metrics
| node_exclusion | precision (original) | precision (optimized) | recall (original) | recall (optimized) | f1 (original) | f1 (optimized) | accuracy (original) | accuracy (optimized) | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
['layernorm', 'gelu', 'residual', 'gather', 'softmax'] | | | 0.936 | 0.904 | | | 0.944 | 0.921 | | | 0.940 | 0.912 | | | 0.988 | 0.984 |
[] | | | 0.936 | 0.065 | | | 0.944 | 0.243 | | | 0.940 | 0.103 | | | 0.988 | 0.357 |
Time metrics
Time benchmarks were run for 15 seconds per config.
Below, time metrics for batch size = 4, input length = 64.
| node_exclusion | latency_mean (original, ms) | latency_mean (optimized, ms) | throughput (original, /s) | throughput (optimized, /s) | ||
|---|---|---|---|---|---|---|
['layernorm', 'gelu', 'residual', 'gather', 'softmax'] | | | 103.46 | 53.77 | | | 9.67 | 18.60 |
[] | | | 90.62 | 65.86 | | | 11.07 | 15.20 |
- Task
- Token Classification
- Created
- Jul 13, 2022
- Updated
- Jul 13, 2022