Downloads · 30 days
58
2% of all-time downloads
ODM_1Bparams_50Btokens_seqlen2048
alon-albalak/ODM_1Bparams_50Btokens_seqlen2048
ODM_1Bparams_50Btokens_seqlen2048 is a feature extraction model from alon-albalak. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as openrail.
This is a GPT-NeoX model trained on 50 billion tokens from The Pile, using the Online Data Mixing method.
All-time downloads
3.9K
Public
Parameters
976M
7.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin3.7 GB · 50%
Parameter types
How the weights are stored.
F32909M · 93%
Model card
From the Hugging Face model README
This is a GPT-NeoX model trained on 50 billion tokens from The Pile, using the Online Data Mixing method.
The OpenLLM leaderboard won't let me submit my model because the description is too short, so I'm adding more characters to the description in hopes that it will be evaluated.
- Task
- Feature Extraction
- Library
- transformers
- Type
- gpt_neox
- License
- openrail
- Languages
- en
- Created
- Oct 26, 2023
- Updated
- Oct 26, 2023