Downloads · 30 days
9
2% of all-time downloads
mega-150m-arch
Bingsu/mega-150m-arch
mega-150m-arch is a fill-mask model from Bingsu. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as mit.
All-time downloads
459
Public
Repo size
1.8 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.bin452 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- mit
- Model type
- mega
- Access
- Public
- Created
- Apr 11, 2023
- Updated
- Apr 24, 2023
- SHA
- d0dabf14
Try a prompt
Model card
From the Hugging Face model README
===================================================================================================================
Layer (type:depth-idx) Output Shape Param #
===================================================================================================================
MegaForMaskedLM [4, 2048, 50265] --
├─MegaModel: 1-1 [4, 2048, 768] --
│ └─MegaEmbeddings: 2-1 [4, 2048, 768] --
│ │ └─Embedding: 3-1 [4, 2048, 768] 38,603,520
│ └─ModuleList: 2-2 -- --
│ │ └─MegaBlock: 3-2 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-3 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-4 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-5 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-6 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-7 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-8 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-9 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-10 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-11 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-12 [2048, 4, 768] 6,202,626
│ │ └─MegaBlock: 3-13 [2048, 4, 768] 6,202,626
├─Linear: 1-2 [4, 2048, 50265] 38,653,785
===================================================================================================================
Total params: 151,688,817
Trainable params: 151,688,817
Non-trainable params: 0
Total mult-adds (G): 150.35
===================================================================================================================
Input size (MB): 0.07
Forward/backward pass size (MB): 10818.75
Params size (MB): 606.71
Estimated Total Size (MB): 11425.52
===================================================================================================================
- Task
- Fill-Mask
- Library
- transformers
- Type
- mega
- License
- mit
- Created
- Apr 11, 2023
- Updated
- Apr 24, 2023