Downloads · 30 days
684
4% of all-time downloads
i-dot-ai/all-miniLM-L6-v2-UKPGA-6k-finetune
all-miniLM-L6-v2-UKPGA-6k-finetune is a sentence similarity model from i-dot-ai. Use it when you need a score for how close two texts are. It is set up for sentence-transformers.
This is a sentence-transformers model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.
Downloads · 30 days
684
4% of all-time downloads
All-time downloads
16.3K
Public
Parameters
22.7M
90.9 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors90.9 MB · 99%
From the Hugging Face model README
This is a sentence-transformers model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.
<!--- Describe your model here -->This model aims to provide an embedding finetuned to UK legislative language.
The model is used in an i-dot-ai project to give better semantic searching results for legislation.
The model has been finetuned on 6k question answer pairs from UK public general acts. The question and answer pairs can be found here: (i-dot-ai/legislation_2021-2023_qa)
The answers are sections from UK public general acts from 2010-2023. The matching questions are generated by gpt-4o.
The model performs similarly well to text-embedding-3-large. The finetuning has increased performance by 12% against it's base model all-miniLM-L6-v2.
Using this model becomes easy when you have sentence-transformers installed:
pip install -U sentence-transformers
Then you can use the model like this:
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]
model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
The model was evaluated against 1000 question and answer pairs from UKPGA 2010-2023 with questions generated by gpt-4o.
For an automated evaluation of this model, see the Sentence Embeddings Benchmark: https://seb.sbert.net
The model was trained with the parameters:
DataLoader:
torch.utils.data.dataloader.DataLoader of length 1734 with parameters:
{'batch_size': 10, 'sampler': 'torch.utils.data.sampler.SequentialSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
Loss:
sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss with parameters:
{'scale': 20.0, 'similarity_fct': 'cos_sim'}
Parameters of the fit()-Method:
{
"epochs": 2,
"evaluation_steps": 50,
"evaluator": "sentence_transformers.evaluation.InformationRetrievalEvaluator.InformationRetrievalEvaluator",
"max_grad_norm": 1,
"optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
"optimizer_params": {
"lr": 2e-05
},
"scheduler": "WarmupLinear",
"steps_per_epoch": null,
"warmup_steps": 346,
"weight_decay": 0.01
}
SentenceTransformer(
(0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)