neural-chat-7b-v3-1-dare-0.85

uukuguy/neural-chat-7b-v3-1-dare-0.85

neural-chat-7b-v3-1-dare-0.85 is a text generation model from uukuguy. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.

Experiment for DARE(Drop and REscale), most of the delta parameters can be directly set to zeros without affecting the capabilities of SFT LMs and larger models can tolerate a higher proportion of discarded parameters.

Downloads · 30 days

0% of all-time downloads

All-time downloads

37.4K

Public

Repo size

29 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin14.5 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: llama2
Model type: mistral
Access: Public
Created: Nov 20, 2023
Updated: Nov 22, 2023
SHA: 3c15d3e2

Try a prompt

Model card

From the Hugging Face model README

weight_mask_rate: 0.85 / use_weight_rescale: True / mask_stratery: random / scaling_coefficient: 1.0

Model	Average	ARC	HellaSwag	MMLU	TruthfulQA	Winogrande	GSM8K	DROP
Intel/neural-chat-7b-v3-1	59.06	66.21	83.64	62.37	59.65	78.14	19.56	43.84
migtissera/SynthIA-7B-v1.3	57.11	62.12	83.45	62.65	51.37	78.85	17.59	43.76
bhenrym14/mistral-7b-platypus-fp16	56.89	63.05	84.15	64.11	45.07	78.53	17.36	45.92
jondurbin/airoboros-m-7b-3.1.2	56.24	61.86	83.51	61.91	53.75	77.58	13.87	41.2
uukuguy/speechless-code-mistral-orca-7b-v1.0	55.33	59.64	82.25	61.33	48.45	77.51	8.26	49.89
teknium/CollectiveCognition-v1.1-Mistral-7B	53.87	62.12	84.17	62.35	57.62	75.37	15.62	19.85
Open-Orca/Mistral-7B-SlimOrca	53.34	62.54	83.86	62.77	54.23	77.43	21.38	11.2
uukuguy/speechless-mistral-dolphin-orca-platypus-samantha-7b	53.34	64.33	84.4	63.72	52.52	78.37	21.38	8.66
ehartford/dolphin-2.2.1-mistral-7b	53.06	63.48	83.86	63.28	53.17	78.37	21.08	8.19
teknium/CollectiveCognition-v1-Mistral-7B	52.55	62.37	85.5	62.76	54.48	77.58	17.89	7.22
HuggingFaceH4/zephyr-7b-alpha	52.4	61.01	84.04	61.39	57.9	78.61	14.03	9.82
ehartford/samantha-1.2-mistral-7b	52.16	64.08	85.08	63.91	50.4	78.53	16.98	6.13

Task: Text Generation
Library: transformers
Type: mistral
License: llama2
Created: Nov 20, 2023
Updated: Nov 22, 2023