Downloads · 30 days
122
3% of all-time downloads
Roflmax/bge-m3-legal-ru-cocktail-40-60
bge-m3-legal-ru-cocktail-40-60 is a sentence similarity model from Roflmax. Use it when you need a score for how close two texts are. It is set up for sentence-transformers. The card lists the license as mit.
🏆 Top-performing Russian legal document embedding model with 91.79% Recall@5
Downloads · 30 days
122
3% of all-time downloads
All-time downloads
4.2K
Public
Parameters
568M
2.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.3 GB · 99%
From the Hugging Face model README
🏆 Top-performing Russian legal document embedding model with 91.79% Recall@5
This model is created using LM-Cocktail weight interpolation technique, combining two state-of-the-art Russian legal embedding models:
Synergistic Effect: The cocktail model outperforms both parent models through optimal weight combination!
| Metric | Score | Rank |
|---|---|---|
| Recall@1 | 76.66% | #1 |
| Recall@5 | 91.79% | 🥇 #1 |
| Recall@10 | 94.85% | #1 |
| Dataset | Recall@1 | Recall@5 | Recall@10 | Description |
|---|---|---|---|---|
| court_law | 66.01% | 86.47% | 91.08% | Court decisions and rulings |
| other_law | 90.44% | 95.80% | 96.97% | Federal laws and codes |
| reg_law | 75.52% | 93.09% | 96.49% | Regional legislation |
| Average | 76.66% | 91.79% | 94.85% | Across all domains |
| Model | Recall@5 | Improvement |
|---|---|---|
| Cocktail 40/60 (this model) | 91.79% | Baseline |
| bge-m3-russian-legal | 91.43% | +0.36% |
| bge-m3-legal-ru-updata | 91.28% | +0.51% |
The cocktail demonstrates synergistic effect - it outperforms both parent models!
pip install -U sentence-transformers
from sentence_transformers import SentenceTransformer
# Load the model
model = SentenceTransformer("Roflmax/bge-m3-legal-ru-cocktail-40-60")
model.max_seq_length = 512 # Optimized for speed
# Example: Semantic search in legal documents
query = "Какое наказание предусмотрено за управление транспортным средством в состоянии опьянения?"
documents = [
"Статья 264.1 УК РФ. Нарушение правил дорожного движения лицом, подвергнутым административному наказанию...",
"КоАП РФ Статья 12.8. Управление транспортным средством водителем, находящимся в состоянии опьянения...",
"Статья 228 УК РФ. Незаконные приобретение, хранение, перевозка, изготовление..."
]
# Encode
query_embedding = model.encode(query, normalize_embeddings=True)
doc_embeddings = model.encode(documents, normalize_embeddings=True)
# Calculate similarity
from sklearn.metrics.pairwise import cosine_similarity
similarities = cosine_similarity([query_embedding], doc_embeddings)[0]
# Get top results
top_indices = similarities.argsort()[::-1]
for idx in top_indices:
print(f"Score: {similarities[idx]:.4f} | {documents[idx][:100]}...")
from sentence_transformers import SentenceTransformer
import numpy as np
model = SentenceTransformer("Roflmax/bge-m3-legal-ru-cocktail-40-60")
model.max_seq_length = 512
# Batch encode documents
documents = [
"Первый документ...",
"Второй документ...",
# ... more documents
]
# Process in batches for efficiency
embeddings = model.encode(
documents,
batch_size=32,
normalize_embeddings=True,
show_progress_bar=True
)
print(f"Generated {len(embeddings)} embeddings of dimension {embeddings.shape[1]}")
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("Roflmax/bge-m3-legal-ru-cocktail-40-60")
# Your corpus
corpus = [
"Документ 1: содержание...",
"Документ 2: содержание...",
# ... more documents
]
# Encode corpus once
corpus_embeddings = model.encode(corpus, convert_to_tensor=True, normalize_embeddings=True)
# Query
query = "Ваш поисковый запрос"
query_embedding = model.encode(query, convert_to_tensor=True, normalize_embeddings=True)
# Search
hits = util.semantic_search(query_embedding, corpus_embeddings, top_k=5)[0]
# Display results
for hit in hits:
print(f"Score: {hit['score']:.4f} | {corpus[hit['corpus_id']][:100]}...")
Unlike some BGE models, this model does NOT require query/passage prefixes. Simply encode your text directly:
# ✅ Correct - no prefix needed
embedding = model.encode("Ваш текст")
# ❌ Not needed
embedding = model.encode("Represent this sentence for searching relevant passages: Ваш текст")
The model is optimized for 512 tokens:
For longer documents, consider chunking:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("Roflmax/bge-m3-legal-ru-cocktail-40-60")
model.max_seq_length = 512
# Split long document into chunks
def chunk_text(text, max_length=2000):
# Simple character-based chunking
return [text[i:i+max_length] for i in range(0, len(text), max_length)]
long_document = "Очень длинный документ..."
chunks = chunk_text(long_document)
chunk_embeddings = model.encode(chunks, normalize_embeddings=True)
# Use average embedding for the whole document
import numpy as np
document_embedding = np.mean(chunk_embeddings, axis=0)
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'XLMRobertaModel'})
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True})
(2): Normalize()
)
This model was created using LM-Cocktail technique (paper), which performs linear weight interpolation between two fine-tuned models:
W_cocktail = 0.4 × W_russian-legal + 0.6 × W_updata
bge-m3-russian-legal (40% weight)
bge-m3-legal-ru-updata (60% weight)
The 40/60 weight ratio was selected based on comprehensive benchmarking:
Find relevant laws, court decisions, and regulations based on natural language queries.
Classify legal documents by type, jurisdiction, or topic using embedding similarity.
Identify similar or duplicate legal documents in large corpora.
Build semantic search systems for legal Q&A applications.
Group related legal documents for analysis and organization.
If you use this model, please cite:
@misc{bge-m3-legal-ru-cocktail,
title={BGE-M3 Legal RU Cocktail: LM-Cocktail Weight Interpolation for Russian Legal Embeddings},
author={Roflmax},
year={2025},
publisher={HuggingFace},
howpublished={\url{https://huggingface.co/Roflmax/bge-m3-legal-ru-cocktail-40-60}}
}
Also cite the parent models:
And the LM-Cocktail paper:
@article{jiang2023lmcocktail,
title={LM-Cocktail: Resilient Tuning of Language Models via Model Merging},
author={Jiang, Shitao and others},
journal={arXiv preprint arXiv:2311.13534},
year={2023}
}
MIT License - free for commercial and non-commercial use.
Note: This model achieves state-of-the-art performance on Russian legal document retrieval tasks. For best results, use with normalize_embeddings=True and cosine similarity.