Jongbin-kr

llama-3.1-8b-instruct-4x2-moe

Jongbin-kr/llama-3.1-8b-instruct-4x2-moe

llama-3.1-8b-instruct-4x2-moe is a text generation model from Jongbin-kr. Use it when you need the model to write or continue text. It is set up for transformers.

This checkpoint sparse-upcycles every Llama 3.1 8B Instruct decoder MLP into four initially identical experts with dropless top-2 routing.