Downloads · 30 days
0
new-equation-adapter-moe-paper
hayulalab/new-equation-adapter-moe-paper
new-equation-adapter-moe-paper is a machine learning model from hayulalab. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Access
Public
Updated Jul 8, 2026
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.md12.1 KB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Jul 8, 2026
- Updated
- Jul 8, 2026
- SHA
- ff8c385b
Model card
From the Hugging Face model README
The New Equation: Dense AdapterMoE — Dynamic Expert Routing on Consumer Hardware — Hayula Research
Hayula AI Lab
Abstract
Mixture-of-Experts models achieve efficient scaling by routing tokens to specialized sub-networks. However, traditional MoE requires training from scratch or expensive full-model fine-tuning. We propose Dense AdapterMoE: a hybrid architecture combining a frozen 7-8B base language model with 10-20 dynamically routed LoRA expert adapters, all loaded simultaneously on consumer hardware (Apple M2 Ultra, 192GB unified memory). Each expert adapter occupies only 37MB (2 million parameters at rank 8), a
Files
| File | Description |
|---|---|
paper.md | Full paper (Markdown) |
README.md | This model card |
Citation
@techreport{hayulalab2026newequationadaptermoe,
title={The New Equation: Dense AdapterMoE — Dynamic Expert Routing on Consumer Hardware — Hayula Research},
author={Hayula AI Lab},
year={2026},
url={https://huggingface.co/hayulalab/new-equation-adapter-moe-paper}
}
hayulalab — Open Source AI Research
- License
- mit
- Created
- Jul 8, 2026
- Updated
- Jul 8, 2026