Downloads · 30 days
6
21% of all-time downloads
cat-emb-4-128
bwang0911/cat-emb-4-128
cat-emb-4-128 is a machine learning model from bwang0911. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
A set of embedding model trained for study embedding quality vs model architecture (width/depth) given a size constraint (12M params).
All-time downloads
29
Public
Repo size
28.7 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin19.2 MB · 99%
At a glance
- Library
- transformers
- License
- apache-2.0
- Model type
- bert
- Access
- Public
- Created
- Jun 14, 2024
- Updated
- Jun 14, 2024
- SHA
- d53941e1
Model card
From the Hugging Face model README
Cat Embeddings
A set of embedding model trained for study embedding quality vs model architecture (width/depth) given a size constraint (12M params).
- cat-emb-2-128: 2 layers/hidden size 128/4.4m
- cat-emb-4-128: 4 layers/H 128/4.8m
- cat-emb-8-128: 8 layers/H 128/5.6m
- cat-emb-12-128: 12 layers/H 128/6.4m
- cat-emb-2-256: 2 layers/H 256/9.7m
- cat-emb-4-256: 4 layers/H 256/11.3m
Training
- stage 1: seq 192, batch size 2048, 50k steps, sentence pairs.
- stage 2: seq 512, batch size 64, 5k steps, sentence triplets.
Perf
- Library
- transformers
- Type
- bert
- License
- apache-2.0
- Created
- Jun 14, 2024
- Updated
- Jun 14, 2024