leonzhou286

llama3_8b_instruct_moe

leonzhou286/llama3_8b_instruct_moe

llama3_8b_instruct_moe is a machine learning model from leonzhou286. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.

Llama 3 8b Instruct base model converted to MOE style by randomly partitioning the FFN layers of each decoder layer into 8 experts of the same size. Weights are directly taken from the llama3 instruct base model.