Downloads · 30 days
83
0% of all-time downloads
SOLARC-MOE-10.7Bx4
DopeorNope/SOLARC-MOE-10.7Bx4
SOLARC-MOE-10.7Bx4 is a text generation model from DopeorNope. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.
All-time downloads
41.1K
Public
Parameters
36.1B
144 GB on disk
Likes
9
Public
Repo makeup
Click a slice to open those files.
.safetensors144 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-sa-4.0
- Model type
- mixtral
- Access
- Public
- Created
- Dec 27, 2023
- Updated
- Jan 15, 2024
- SHA
- 974d9eec
Model card
From the Hugging Face model README
The license is cc-by-nc-sa-4.0.
🐻❄️SOLARC-MOE-10.7Bx4🐻❄️
Model Details
Model Developers Seungyoo Lee(DopeorNope)
I am in charge of Large Language Models (LLMs) at Markr AI team in South Korea.
Input Models input text only.
Output Models generate text only.
Model Architecture
SOLARC-MOE-10.7Bx4 is an auto-regressive language model based on the SOLAR architecture.
Base Model
kyujinpy/Sakura-SOLAR-Instruct
Weyaxi/SauerkrautLM-UNA-SOLAR-Instruct
VAGOsolutions/SauerkrautLM-SOLAR-Instruct
fblgit/UNA-SOLAR-10.7B-Instruct-v1.0
Implemented Method
I have built a model using the Mixture of Experts (MOE) approach, utilizing each of these models as the base.
Implementation Code
Load model
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "DopeorNope/SOLARC-MOE-10.7Bx4"
OpenOrca = AutoModelForCausalLM.from_pretrained(
repo,
return_dict=True,
torch_dtype=torch.float16,
device_map='auto'
)
OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- cc-by-nc-sa-4.0
- Languages
- ko
- Created
- Dec 27, 2023
- Updated
- Jan 15, 2024