Downloads · 30 days
34
1% of all-time downloads
xlm-roberta-longformer-base-16384
hyperonym/xlm-roberta-longformer-base-16384
xlm-roberta-longformer-base-16384 is a feature extraction model from hyperonym. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as mit.
xlm-roberta-longformer is a multilingual Longformer initialized with XLM-RoBERTa's weights without further pretraining. It is intended to be fine-tuned on a downstream task.
All-time downloads
5.9K
Public
Repo size
1.3 GB
Likes
20
Public
Repo makeup
Click a slice to open those files.
.h51.2 GB · 99%
At a glance
- Task
- Feature Extraction
- Library
- transformers
- License
- mit
- Model type
- longformer
- Access
- Public
- Created
- Jan 29, 2023
- Updated
- Apr 29, 2023
- SHA
- 4e786532
Model card
From the Hugging Face model README
xlm-roberta-longformer-base-16384
xlm-roberta-longformer is a multilingual Longformer initialized with XLM-RoBERTa's weights without further pretraining. It is intended to be fine-tuned on a downstream task.
| Model | attention_window | hidden_size | num_hidden_layers | model_max_length |
|---|---|---|---|---|
| base | 256 | 768 | 12 | 16384 |
| large | 512 | 1024 | 24 | 16384 |
Framework versions
- Transformers 4.26.0
- TensorFlow 2.11.0
- Tokenizers 0.13.2
- Task
- Feature Extraction
- Library
- transformers
- Type
- longformer
- License
- mit
- Languages
- multilingual, af, am, ar, as, az
- Created
- Jan 29, 2023
- Updated
- Apr 29, 2023