Skip to content

chestnutlzj

MoE-Qwen-4x1.8B-pretrain-50000-ckpt

chestnutlzj/MoE-Qwen-4x1.8B-pretrain-50000-ckpt

MoE-Qwen-4x1.8B-pretrain-50000-ckpt is a text generation model from chestnutlzj. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

This is a MoE model project constructed based on the Qwen 1.8B model. In this project, we concatenated 4 original models and trained them using special training methods.

Downloads · 30 days

241

7% of all-time downloads

All-time downloads

3.2K

Public

Parameters

4.3B

8.5 GB on disk

Likes

1

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors8.5 GB · 100%

Parameter types

How the weights are stored.

BF164.3B · 100%

Task
Text Generation
Library
transformers
Type
mixtral
License
apache-2.0
Languages
zh
Created
Jan 10, 2024
Updated
Jan 10, 2024