gpt2-decomposed

Lumi-node/gpt2-decomposed

gpt2-decomposed is a machine learning model from Lumi-node. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for model-garage. The card lists the license as apache-2.0.

Full component-level decomposition of GPT-2 (124M parameters) using Model Garage.

Downloads · 30 days

Access

Public

Updated Mar 24, 2026

Repo size

838 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.pt993 MB · 100%

At a glance

Library: model-garage
License: apache-2.0
Access: Public
Created: Mar 24, 2026
Updated: Mar 24, 2026
SHA: 6b16e1ef

Model card

From the Hugging Face model README

GPT-2 Decomposed — Model Garage

Full component-level decomposition of GPT-2 (124M parameters) using Model Garage.

What's Here

64 individually extracted nn.Module components:

Component Type	Count	Dimensions
Attention (self_attn)	12	768d, 12 heads, 64 head_dim
Feed-Forward (mlp)	12	768→3072→768
Layer Norm (ln_1, ln_2)	24	768d
Full Layers	12	768d
Embeddings	2	token (50257→768), position (1024→768)
Output Head	1	768→50257
Final Norm	1	768d

Usage

from model_garage.extract.pytorch import PyTorchExtractor

extractor = PyTorchExtractor("gpt2")
extractor.load_model()

# Extract any component
attn = extractor.extract_component("self_attention", layer_idx=6)
ffn = extractor.extract_component("feed_forward", layer_idx=6)

# Test in isolation
from model_garage.extract.pytorch import ComponentTester
tester = ComponentTester()
print(tester.test_attention(attn))

Install

pip install model-garage