yi-6b-w2a16g32

GreenBitAI/yi-6b-w2a16g32

yi-6b-w2a16g32 is a text generation model from GreenBitAI. Use it when you need the model to write or continue text. It is set up for transformers.

Downloads · 30 days

12% of all-time downloads

All-time downloads

131

Public

Parameters

1.1B

2.7 GB on disk

Likes

Public

Repo makeup

Click a slice to open those files.

.safetensors2.7 GB · 100%

How the weights are stored.

F16546M · 51%

From the Hugging Face model README

This is GreenBitAI's pretrained 2-bit Yi 6B model with extreme compression yet still strong performance.

Please refer to our Github page for the code to run the model and more information.

Task	Metric	FP16	Yi-6B w4a16g32	Yi-6B w2a16g8	Yi-6B w2a16g32
Openbookqa	acc	0.314	0.324	0.26	0.228
	ac_norm	0.408	0.42	0.394	0.352
arc_challenge	acc	0.462	0.4573	0.4082	0.337
	ac_norm	0.504	0.483	0.4249	0.3523
hellawswag	acc	0.553	0.5447	0.5083	0.4326
	ac_norm	0.749	0.7327	0.6909	0.58
piqa	acc	0.777	0.7709	0.7535	0.7051
	ac_norm	0.787	0.7894	0.7655	0.7143
arc_easy	acc	0.777	0.7697	0.7373	0.6523
	ac_norm	0.774	0.7659	0.7314	0.6115
Winogrande	acc	0.707	0.7095	0.6803	0.6219
boolq	acc	0.755	0.7648	0.7507	0.732
truthfulqa_mc	mc1	0.29	0.2729	0.2753	0.219
	mc2	0.419	0.4033	0.4156	0.3479
anli_r1	acc	0.423	0.416	0.383	0.38
anli_r2	acc	0.409	0.409	0.387	0.374
anli_r3	acc	0.411	0.393	0.38	0.3475
wic	acc	0.529	0.545	0.515	0.5
rte	acc	0.685	0.7039	0.7436	0.6787
record	f1	0.904	0.9011	0.8906	0.8521
	em	0.8962	0.8927	0.8819	0.8429
Average		0.596	0.5937	0.5703	0.517
wikitext2 (2048)	ppl	5.841	6.01	6.57	8.06
ptb (2048)	ppl	18.93	19.76	25.9	35.3
Model Size	GiB	12.12	3.76	3.09	2.5