Downloads · 30 days
11
4% of all-time downloads
Phi3Mix
fukayatti/Phi3Mix
Phi3Mix is a text generation model from fukayatti. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Phi3Mix is a Mixture of Experts (MoE) made with the following models using Phi3LazyMergekit: microsoft/Phi-3-small-128k-instruct Rakuten/RakutenAI-7B
All-time downloads
288
Public
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.py85.3 KB · 92%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- phi3
- Access
- Public
- Created
- Jun 14, 2024
- Updated
- Jun 14, 2024
- SHA
- f585c88e
Try a prompt
Base models
Model card
From the Hugging Face model README
Phi3Mix
Phi3Mix is a Mixture of Experts (MoE) made with the following models using Phi3_LazyMergekit:
🧩 Configuration
base_model: microsoft/Phi-3-small-128k-instruct
gate_mode: cheap_embed
experts_per_token: 1
dtype: float16
experts:
- source_model: microsoft/Phi-3-small-128k-instruct
positive_prompts: ["research, logic, math, science"]
- source_model: Rakuten/RakutenAI-7B
positive_prompts: ["creative, art"]
💻 Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = "fukayatti/Phi3Mix"
tokenizer = AutoTokenizer.from_pretrained(model)
model = AutoModelForCausalLM.from_pretrained(
model,
trust_remote_code=True,
)
prompt="How many continents are there?"
input = f"<|system|>You are a helpful AI assistant.<|end|><|user|>{prompt}<|assistant|>"
tokenized_input = tokenizer.encode(input, return_tensors="pt")
outputs = model.generate(tokenized_input, max_new_tokens=128, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(tokenizer.decode(outputs[0]))
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- apache-2.0
- Created
- Jun 14, 2024
- Updated
- Jun 14, 2024