Downloads · 30 days
158
1% of all-time downloads
klue-cross-encoder-v1
bongsoo/klue-cross-encoder-v1
klue-cross-encoder-v1 is a text classification model from bongsoo. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
- klue/bert-base 모델을 훈련시켜 cross-encoder로 파인튜닝한 모델 - This model was trained using SentenceTransformers Cross-Encoder class.
All-time downloads
16.4K
Public
Repo size
885 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin443 MB · 100%
At a glance
- Task
- Text Classification
- Library
- transformers
- License
- apache-2.0
- Model type
- bert
- Access
- Public
- Created
- Jan 30, 2023
- Updated
- Jan 30, 2023
- SHA
- e7287561
Model card
From the Hugging Face model README
klue-cross-encoder-v1
- klue/bert-base 모델을 훈련시켜 cross-encoder로 파인튜닝한 모델
- This model was trained using SentenceTransformers Cross-Encoder class.
Training
-
sts(10)-sts(10)훈련 시킴
-
STS : seed=111,epoch=10, lr=1e-4, eps=1e-6, warm_step=10%, max_seq_len=128, train_batch=128(small 모델=32) (albert 13m/7G) 훈련코드
| 모델 | korsts | klue-sts | glue(stsb) | stsb_multi_mt(en) |
|---|---|---|---|---|
| albert-small-kor-cross-encoder-v1 | 0.8455 | 0.8526 | 0.8513 | 0.7976 |
| klue-cross-encoder-v1 | 0.8262 | 0.8833 | 0.8512 | 0.7889 |
| kpf-cross-encoder-v1 | 0.8799 | 0.9133 | 0.8626 | 0.8027 |
Usage and Performance
Pre-trained models can be used like this:
from sentence_transformers import CrossEncoder
model = CrossEncoder('bongsoo/kpf-cross-encoder-v1')
scores = model.predict([('오늘 날씨가 좋다', '오늘 등산을 한다'), ('오늘 날씨가 흐리다', '오늘 비가 내린다')])
print(scores)
[0.10161418 0.45563662]
The model will predict scores for the pairs ('Sentence 1', 'Sentence 2') and ('Sentence 3', 'Sentence 4').
You can use this model also without sentence_transformers and by just using Transformers AutoModel class
- Task
- Text Classification
- Library
- transformers
- Type
- bert
- License
- apache-2.0
- Languages
- ko
- Created
- Jan 30, 2023
- Updated
- Jan 30, 2023