finetune-matryoshka-telecom-embeddings

agraharr/finetune-matryoshka-telecom-embeddings

Task: Domain-adapted Sentence Embeddings — Telecom NLI, Retrieval, Similarity

---

Model Overview

This model is a domain-specialized telecom sentence embedding model. It was finetuned from tomaarsen/mpnet-base-nli-matryoshka using Multiple Negatives Ranking Loss (contrastive SFT) over thousands of telecom QA pairs.

• Base: MPNet (Matryoshka sequence pooling)
• Finetune: Hard positive (label=1) sentence pairs from telecom Q&A, standards, research
• Downstream: Semantic search, similarity, clustering, answer retrieval in telecom tasks

---

How to Use (Embedding any sentence)

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("agraharr/finetune-matryoshka-telecom-embeddings")
emb = model.encode(["What is a gNodeB in 5G?"])
# emb is a (1, embedding_dim) numpy array

Batch encoding:

sentences = [
    "What is Open RAN?",
    "Define DSS in telecom",
    "Explain SDN virtualization"
]
vecs = model.encode(sentences)

---

Training details

• Library: sentence-transformers
• Model: tomaarsen/mpnet-base-nli-matryoshka
• Loss: MultipleNegativesRankingLoss
• Positives: Label=1 sentence pairs (QA) from telecom-wide dataset (teleqna_sentence_pairs.tsv)
• Epochs: 4 Batch: 32
• Optimizer: AdamW (default sbert)
• Hardware: a10g-large GPU on Hugging Face Spaces
• Logging: Trackio (space)

Data

• ~46,322 telecom sentence pairs (questions, correct answers)
• Constructed & curated from standards (3GPP, IEEE, industry lexicons, publications)
• All pairs: teleqna_sentence_pairs.tsv

---

Citation

• Base Model: Tijs Tomaarsen. "mpnet-base-nli-matryoshka." Hugging Face. https://huggingface.co/tomaarsen/mpnet-base-nli-matryoshka
• Matryoshka Pooling: Liu, X. et al. "Matryoshka Representations for Efficient and Robust Sentence Embeddings." NeurIPS 2023. arXiv:2310.06665
• Data Curation: agraharr, Telecom QA mining 2024 (see repo for script/data generation)

---

Author & Contact

• Trained/curated by agraharr
• Feedback/issues: https://huggingface.co/agraharr