phi-2-2x4-hf

mzbac/phi-2-2x4-hf

phi-2-2x4-hf is a text generation model from mzbac. Use it when you need the model to write or continue text. It is set up for transformers.

The Moe model was constructed using 4 microsoft/phi-2. Then qlora was applied to all linear layers on WizardLMevolinstruct70k via mlx. The model was created using a script from https://github.com/mzbac/mlx-moe

Downloads · 30 days

9% of all-time downloads

All-time downloads

208

Public

Parameters

7.8B

15.6 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors15.6 GB · 100%

At a glance

Task: Text Generation
Library: transformers
Model type: phi2moe
Access: Public
Created: Feb 16, 2024
Updated: Feb 16, 2024
SHA: 1a6b1d06

Try a prompt

Model card

From the Hugging Face model README

The Moe model was constructed using 4 microsoft/phi-2. Then qlora was applied to all linear layers on WizardLM_evol_instruct_70k via mlx. The model was created using a script from https://github.com/mzbac/mlx-moe

Evaluation

MMLU

mzbac/phi-2-2x4-hf

Groups	Version	Filter	Metric	Value		Stderr
- humanities	N/A	none	acc	0.5970	±	0.0245
- other	N/A	none	acc	0.5760	±	0.0311
- social_sciences	N/A	none	acc	0.6610	±	0.0284
- stem	N/A	none	acc	0.4738	±	0.0379

microsoft/phi-2

Groups	Version	Filter	Metric	Value		Stderr
- humanities	N/A	none	acc	0.6026	±	0.0243
- other	N/A	none	acc	0.5827	±	0.0310
- social_sciences	N/A	none	acc	0.6440	±	0.0289
- stem	N/A	none	acc	0.4721	±	0.0377

Example

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mzbac/phi-2-2x4-hf"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)

text = "Instruct: how backpropagation works.\nOutput:"
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Task: Text Generation
Library: transformers
Type: phi2moe
Created: Feb 16, 2024
Updated: Feb 16, 2024