Orca-2-13b-W8A8-Dynamic-Per-Token

noneUsername/Orca-2-13b-W8A8-Dynamic-Per-Token

Orca-2-13b-W8A8-Dynamic-Per-Token is a machine learning model from noneUsername. Use it for the machine learning task on the model card, and read the license before you ship it in a product.

vllm (pretrained=/root/autodl-tmp/Orca-2-13b,addbostoken=true,tensorparallelsize=2,maxmodellen=2048,dtype=bfloat16), genkwargs: (None), limit: 250.0, numfewshot: 5, batchsize: 1

Downloads · 30 days

25% of all-time downloads

All-time downloads

Public

Parameters

13B

13.3 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors13.3 GB · 100%

Parameter types

How the weights are stored.

I812.7B · 97%

Base models

microsoft/Orca-2-13b

Model card

From the Hugging Face model README

vllm (pretrained=/root/autodl-tmp/Orca-2-13b,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: 1

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.520	±	0.0317
		strict-match	5	exact_match	↑	0.504	±	0.0317

The above are the results of the original model. The results of this model have a deviation of only 0.01.

vllm (pretrained=/root/autodl-tmp/Orca-2-13b,add_bos_token=true,max_model_len=2048,dtype=float32), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.496	±	0.0317
		strict-match	5	exact_match	↑	0.488	±	0.0317

vllm (pretrained=/root/autodl-tmp/Orca-2-13b,add_bos_token=true,max_model_len=2048,dtype=float16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.508	±	0.0317
		strict-match	5	exact_match	↑	0.496	±	0.0317

vllm (pretrained=/root/autodl-tmp/Orca-2-13b,add_bos_token=true,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.496	±	0.0317
		strict-match	5	exact_match	↑	0.488	±	0.0317

vllm (pretrained=/root/autodl-tmp/Orca-2-13b,add_bos_token=true,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 500.0, num_fewshot: 5, batch_size: auto

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.504	±	0.0224
		strict-match	5	exact_match	↑	0.480	±	0.0224

vllm (pretrained=/root/autodl-tmp/Orca-2-13b,add_bos_token=true,max_model_len=800,dtype=bfloat16), gen_kwargs: (None), limit: 15.0, num_fewshot: None, batch_size: auto

Groups	Version	Filter	Metric		Value		Stderr
mmlu	2	none	acc	↑	0.5766	±	0.0158
- humanities	2	none	acc	↑	0.6359	±	0.0311
- other	2	none	acc	↑	0.6513	±	0.0344
- social sciences	2	none	acc	↑	0.6167	±	0.0350
- stem	2	none	acc	↑	0.4596	±	0.0274

vllm (pretrained=/root/autodl-tmp/Orca-2-13b-W8A8,add_bos_token=true,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.508	±	0.0317
		strict-match	5	exact_match	↑	0.496	±	0.0317

vllm (pretrained=/root/autodl-tmp/Orca-2-13b-W8A8,add_bos_token=true,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 500.0, num_fewshot: 5, batch_size: auto

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.500	±	0.0224
		strict-match	5	exact_match	↑	0.486	±	0.0224

vllm (pretrained=/root/autodl-tmp/Orca-2-13b-W8A8,add_bos_token=true,max_model_len=800,dtype=bfloat16), gen_kwargs: (None), limit: 15.0, num_fewshot: None, batch_size: auto

Groups	Version	Filter	Metric		Value		Stderr
mmlu	2	none	acc	↑	0.5708	±	0.0160
- humanities	2	none	acc	↑	0.6256	±	0.0319
- other	2	none	acc	↑	0.6513	±	0.0346
- social sciences	2	none	acc	↑	0.6222	±	0.0353
- stem	2	none	acc	↑	0.4456	±	0.0277

Type: llama
Created: Oct 6, 2024
Updated: Feb 28, 2025