Skip to content

garagelm

hybrid-gpt-moe-284m-a114m

garagelm/hybrid-gpt-moe-284m-a114m

hybrid-gpt-moe-284m-a114m is a text generation model from garagelm. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

A 284M-parameter (114M active) research model: the lab's hybrid local+global attention stack with every FFN replaced by a top-2 mixture of 8 SwiGLU experts. Trained on one Apple M4 Pro Mac mini on 100M tokens of refin…

Downloads · 30 days

22

4% of all-time downloads

All-time downloads

613

Public

Parameters

285M

569 MB on disk

Likes

0

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors569 MB · 99%

Parameter types

How the weights are stored.

BF16284M · 100%

Try a prompt

Task
Text Generation
Library
transformers
Type
moe_gpt
License
apache-2.0
Languages
en
Created
Aug 22, 2026
Updated
Aug 22, 2026