Downloads · 30 days
16
44% of all-time downloads
sciembed-ctx-8192
J0nasW/sciembed-ctx-8192
sciembed-ctx-8192 is a feature extraction model from J0nasW. Use it when you need embeddings to search or compare text. It is set up for sentence-transformers. The card lists the license as mit.
Long-context variant (maxseqlength=8192). Recommended for long scientific inputs; best within-recipe scores on the Body-Fact Retrieval probe and LongEmbed.
All-time downloads
36
Public
Parameters
149M
596 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors596 MB · 99%
At a glance
- Task
- Feature Extraction
- Library
- sentence-transformers
- License
- mit
- Model type
- modernbert
- Access
- Public
- Created
- Aug 27, 2026
- Updated
- Aug 27, 2026
- SHA
- 44c3280c
Base models
Model card
From the Hugging Face model README
SciEmbed-CTX-8192
Long-context variant (max_seq_length=8192). Recommended for long scientific inputs; best within-recipe scores on the Body-Fact Retrieval probe and LongEmbed.
A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the SciEmbed release (Findings of EMNLP 2026).
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("J0nasW/sciembed-ctx-8192")
emb = model.encode(["citation-context supervision for scientific embeddings"],
normalize_embeddings=True)
- Context length: 8192 tokens
- Pooling: mean · Output dim: 768 (Matryoshka-truncatable to 512/256/128)
- License: MIT
Citation
See the repository README. Paper: SciEmbed: Citation-Context Supervision for Scientific Document Embeddings, Findings of the Association for Computational Linguistics: EMNLP 2026.
- Task
- Feature Extraction
- Library
- sentence-transformers
- Type
- modernbert
- License
- mit
- Languages
- en
- Created
- Aug 27, 2026
- Updated
- Aug 27, 2026