Qwen-1.5-32B-layer-mix-bpw-2.2

GreenBitAI/Qwen-1.5-32B-layer-mix-bpw-2.2

Qwen-1.5-32B-layer-mix-bpw-2.2 is a text generation model from GreenBitAI. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

This is GreenBitAI's pretrained low-bit LLMs with extreme compression yet still strong performance.

Downloads · 30 days

20% of all-time downloads

All-time downloads

Public

Parameters

4.6B

13.3 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors13.3 GB · 100%

Parameter types

How the weights are stored.

F162.5B · 55%

Try a prompt

Model card

From the Hugging Face model README

GreenBit LLMs

This is GreenBitAI's pretrained low-bit LLMs with extreme compression yet still strong performance.

Please refer to our Github page for the code to run the model and more information.

Zero-shot Evaluation

We evaluate the zero-shot ability of low-bit quantized Qwen1.5 models using the llm_eval library and list the results below:

Repository (Qwen Family)	Avg Acc.	OpenBQ	ARC-E	Winogr.	HellaS.	ARC-C	PIQA	BoolQ	RACE	ANLI-R1	ANLI-R2	ANLI-R3	WiC
`Qwen-1.5-0.5B-layer-mix-bpw-2.2`	0.398	0.170	0.443	0.527	0.332	0.238	0.634	0.620	0.318	0.332	0.338	0.330	0.500
`Qwen-1.5-0.5B-layer-mix-bpw-2.5`	0.394	0.170	0.514	0.541	0.337	0.232	0.637	0.496	0.318	0.316	0.358	0.326	0.490
`Qwen-1.5-0.5B-layer-mix-bpw-3.0`	0.407	0.198	0.533	0.536	0.348	0.234	0.671	0.552	0.323	0.330	0.333	0.335	0.495
`Qwen-1.5-1.8B-layer-mix-bpw-2.2`	0.415	0.218	0.539	0.586	0.392	0.260	0.678	0.622	0.333	0.333	0.333	0.336	0.464
`Qwen-1.5-1.8B-layer-mix-bpw-2.5`	0.423	0.222	0.592	0.585	0.406	0.267	0.695	0.629	0.336	0.314	0.339	0.361	0.507
`Qwen-1.5-1.8B-layer-mix-bpw-3.0`	0.438	0.246	0.576	0.563	0.413	0.277	0.694	0.645	0.352	0.323	0.336	0.343	0.492
`Qwen-1.5-4B-layer-mix-bpw-2.2`	0.480	0.254	0.663	0.623	0.463	0.339	0.712	0.718	0.349	0.326	0.355	0.384	0.513
`Qwen-1.5-4B-layer-mix-bpw-2.5`	0.490	0.266	0.677	0.629	0.473	0.365	0.732	0.717	0.351	0.372	0.352	0.360	0.502
`Qwen-1.5-4B-layer-mix-bpw-3.0`	0.502	0.268	0.678	0.642	0.494	0.358	0.755	0.757	0.380	0.395	0.395	0.392	0.519
`Qwen-1.5-7B-layer-mix-bpw-2.2`	0.513	0.278	0.669	0.654	0.504	0.389	0.741	0.759	0.376	0.383	0.410	0.403	0.517
`Qwen-1.5-7B-layer-mix-bpw-2.5`	0.520	0.294	0.705	0.650	0.520	0.387	0.750	0.769	0.371	0.445	0.424	0.398	0.564
`Qwen-1.5-7B-layer-mix-bpw-3.0`	0.531	0.292	0.713	0.654	0.545	0.405	0.764	0.807	0.383	0.424	0.393	0.414	0.627
`Qwen-1.5-14B-layer-mix-bpw-2.5`	0.553	0.318	0.727	0.682	0.564	0.413	0.775	0.792	0.390	0.472	0.434	0.446	0.623
`Qwen-1.5-32B-layer-mix-bpw-3.0`	0.599	0.346	0.775	0.722	0.620	0.492	0.807	0.853	0.444	0.515	0.494	0.478	0.642

Task: Text Generation
Library: transformers
Type: qwen2
License: apache-2.0
Created: Apr 10, 2024
Updated: Apr 30, 2024