embedding-model-midterm-submission-updated

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2 <!-- at revision fa97f6e7cb1a59073dff9e6b13e2715cf7475ac9 -->
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dabraldeepti25/embedding-model-midterm-submission-updated")
# Run inference
sentences = [
    'Comprehensive Flight Analysis: NYC to ROM\n            \n            Route Overview:\n            \n    Route Characteristics:\n    - Distance and typical duration\n    - Common connection points\n    - Seasonal weather impact\n    - Time zone considerations\n    \n    Market Analysis:\n    - Popular travel periods\n    - Price fluctuation patterns\n    - Competing airlines\n    - Alternative routes\n    \n    Operational Considerations:\n    - Aircraft types commonly used\n    - Typical delays and causes\n    - Seasonal performance metrics\n    - Airport congestion analysis\n    \n            \n            Pricing Information:\n            Base Fare: 184.00\n            Total Price: 333.65 EUR\n            \n            Detailed Flight Information:\n            \n                    Flight Segment Analysis:\n                    Carrier: TP 204\n                    Equipment: 32Q\n                    \n                    Departure Details:\n                    Airport: EWR\n                    Terminal: B\n                    Time: 2025-03-27T00:55:00\n                    \n                    Arrival Details:\n                    Airport: LIS\n                    Terminal: 1\n                    Time: 2025-03-27T11:40:00\n                    \n                    Operational Information:\n                    - Aircraft specifications\n                    - Typical on-time performance\n                    - Seasonal reliability metrics\n                    \n                    Flight Segment Analysis:\n                    Carrier: TP 838\n                    Equipment: 32N\n                    \n                    Departure Details:\n                    Airport: LIS\n                    Terminal: 1\n                    Time: 2025-03-27T20:00:00\n                    \n                    Arrival Details:\n                    Airport: FCO\n                    Terminal: 1\n                    Time: 2025-03-28T00:05:00\n                    \n                    Operational Information:\n                    - Aircraft specifications\n                    - Typical on-time performance\n                    - Seasonal reliability metrics\n                    \n            \n            Route Market Analysis:\n            - Historical price trends\n            - Peak travel periods\n            - Alternative routing options\n            - Alliance and codeshare details\n            \n            Airport Information:\n            \n    Airport: NYC\n    \n    Terminal Information:\n    - Layout and facilities\n    - Transfer processes\n    - Security procedures\n    - Lounges and services\n    \n    Ground Transportation:\n    - Public transit options\n    - Taxi and ride-share\n    - Car rental facilities\n    - Parking services\n    \n    Amenities:\n    - Dining options\n    - Shopping facilities\n    - Business services\n    - Medical facilities\n    \n            \n    Airport: ROM\n    \n    Terminal Information:\n    - Layout and facilities\n    - Transfer processes\n    - Security procedures\n    - Lounges and services\n    \n    Ground Transportation:\n    - Public transit options\n    - Taxi and ride-share\n    - Car rental facilities\n    - Parking services\n    \n    Amenities:\n    - Dining options\n    - Shopping facilities\n    - Business services\n    - Medical facilities\n    \n            \n            Travel Planning Guidelines:\n            - Optimal booking windows\n            - Fare class benefits\n            - Baggage policies\n            - Transit visa requirements\n            - Connection considerations\n            \n            Additional Services:\n            - Available ancillary services\n            - Lounge access details\n            - Special assistance services\n            - Meal and seat selection options',
    "thumb|300x300px|The Colosseum\nTh\n            \n            Nearby Attractions:\n            '''Rome''' (Italian and Latin: ''Roma''), the 'Eternal City', is the capital and largest city of Italy and of the Lazio region. It's the famed city of the Roman Empire, the Seven Hills, ''La Dolce Vita'', the Vatican City and ''Three Coins in the Fountain''. Rome, as a millennia-long centre of power, culture and religion, was the centre of one of the greatest civilisations ever, and has exerted a huge influence over the world in its circa 2500 years of existence.\nthumb|300x300px|The Colosseum\nThe historic centre of the city is a UNESCO World Heritage Site. With wonderful palaces, thousand-year-old churches and basilicas, grand romantic ruins, opulent monuments, ornate statues and graceful fountains, Rome has an immensely rich historical heritage and cosmopolitan atmosphere, making it one of Europe's and the world's most visited, famous, influential and beautiful capitals. Today, Rome has a growing nightlife scene and is also seen as a shopping heaven, being regarded as one of the fashi\n            \n            Local Dining Scene:\n            '''Rome''' (Italian and Latin: ''Roma''), the 'Eternal City', is the capital and largest city of Italy and of the Lazio region. It's the famed city of the Roman Empire, the Seven Hills, ''La Dolce Vita'', the Vatican City and ''Three Coins in the Fountain''. Rome, as a millennia-long centre of power, culture and religion, was the centre of one of the greatest civilisations ever, and has exerted a huge influence over the world in its circa 2500 years of existence.\nthumb|300x300px|The Colosseum\nTh",
    'Comprehensive Flight Analysis: NYC to ROM\n            \n            Route Overview:\n            \n    Route Characteristics:\n    - Distance and typical duration\n    - Common connection points\n    - Seasonal weather impact\n    - Time zone considerations\n    \n    Market Analysis:\n    - Popular travel periods\n    - Price fluctuation patterns\n    - Competing airlines\n    - Alternative routes\n    \n    Operational Considerations:\n    - Aircraft types commonly used\n    - Typical delays and causes\n    - Seasonal performance metrics\n    - Airport congestion analysis\n    \n            \n            Pricing Information:\n            Base Fare: 184.00\n            Total Price: 333.65 EUR\n            \n            Detailed Flight Information:\n            \n                    Flight Segment Analysis:\n                    Carrier: TP 202\n                    Equipment: 32Q\n                    \n                    Departure Details:\n                    Airport: EWR\n                    Terminal: B\n                    Time: 2025-03-27T18:40:00\n                    \n                    Arrival Details:\n                    Airport: LIS\n                    Terminal: 1\n                    Time: 2025-03-28T05:25:00\n                    \n                    Operational Information:\n                    - Aircraft specifications\n                    - Typical on-time performance\n                    - Seasonal reliability metrics\n                    \n                    Flight Segment Analysis:\n                    Carrier: TP 834\n                    Equipment: 32Q\n                    \n                    Departure Details:\n                    Airport: LIS\n                    Terminal: 1\n                    Time: 2025-03-28T11:45:00\n                    \n                    Arrival Details:\n                    Airport: FCO\n                    Terminal: 1\n                    Time: 2025-03-28T15:50:00\n                    \n                    Operational Information:\n                    - Aircraft specifications\n                    - Typical on-time performance\n                    - Seasonal reliability metrics\n                    \n            \n            Route Market Analysis:\n            - Historical price trends\n            - Peak travel periods\n            - Alternative routing options\n            - Alliance and codeshare details\n            \n            Airport Information:\n            \n    Airport: NYC\n    \n    Terminal Information:\n    - Layout and facilities\n    - Transfer processes\n    - Security procedures\n    - Lounges and services\n    \n    Ground Transportation:\n    - Public transit options\n    - Taxi and ride-share\n    - Car rental facilities\n    - Parking services\n    \n    Amenities:\n    - Dining options\n    - Shopping facilities\n    - Business services\n    - Medical facilities\n    \n            \n    Airport: ROM\n    \n    Terminal Information:\n    - Layout and facilities\n    - Transfer processes\n    - Security procedures\n    - Lounges and services\n    \n    Ground Transportation:\n    - Public transit options\n    - Taxi and ride-share\n    - Car rental facilities\n    - Parking services\n    \n    Amenities:\n    - Dining options\n    - Shopping facilities\n    - Business services\n    - Medical facilities\n    \n            \n            Travel Planning Guidelines:\n            - Optimal booking windows\n            - Fare class benefits\n            - Baggage policies\n            - Transit visa requirements\n            - Connection considerations\n            \n            Additional Services:\n            - Available ancillary services\n            - Lounge access details\n            - Special assistance services\n            - Meal and seat selection options',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Semantic Similarity

• Evaluated with <code>EmbeddingSimilarityEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,164 training samples
• Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	<ul><li>min: 92 tokens</li><li>mean: 254.69 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 92 tokens</li><li>mean: 255.18 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 0.0</li><li>mean: 0.15</li><li>max: 0.3</li></ul>

• Samples:

  sentence_0	sentence_1	label
  <code>Comprehensive Flight Analysis: NYC to PAR<br> <br> Route Overview:<br> <br> Route Characteristics:<br> - Distance and typical duration<br> - Common connection points<br> - Seasonal weather impact<br> - Time zone considerations<br> <br> Market Analysis:<br> - Popular travel periods<br> - Price fluctuation patterns<br> - Competing airlines<br> - Alternative routes<br> <br> Operational Considerations:<br> - Aircraft types commonly used<br> - Typical delays and causes<br> - Seasonal performance metrics<br> - Airport congestion analysis<br> <br> <br> Pricing Information:<br> Base Fare: 49.00<br> Total Price: 234.18 EUR<br> <br> Detailed Flight Information:<br> <br> Flight Segment Analysis:<br> Carrier: B6 118<br> Equipment: E90<br> <br> Departure Details:<br> Airport: JFK<br> Terminal: 5<br> ...</code>	<code>Hotel Comprehensive Profile: PREMIER INN CROYDON SOUTH<br> <br> Location Analysis:<br> City: LON<br> Precise Location: 51.36269, <br> -0.07195<br> Country: GB<br> <br> Property Details:<br> Chain: PI<br> Category: Standard Hotel<br> Last Updated: 2023-06-15T10:18:03<br> <br> Neighborhood Overview:<br> <br> Neighborhood Characteristics:<br> - Local atmosphere and vibe<br> - Safety and security assessment<br> - Proximity to business districts<br> - Entertainment and dining options<br> - Cultural attractions nearby<br> - Shopping facilities<br> - Green spaces and recreation<br> <br> Transportation Hub Analysis:<br> - Major transit stations<br> - Bus and tram routes<br> - Taxi availability<br> - Bike-sharing stations<br> <br> Local Life:<br> - Popular local venues<br> - Markets and shopping areas<br> - Cultural institutions<br> - Sports facilities<br> <br> ...</code>	<code>0.09089406418063414</code>
  <code>Comprehensive Flight Analysis: NYC to PAR<br> <br> Route Overview:<br> <br> Route Characteristics:<br> - Distance and typical duration<br> - Common connection points<br> - Seasonal weather impact<br> - Time zone considerations<br> <br> Market Analysis:<br> - Popular travel periods<br> - Price fluctuation patterns<br> - Competing airlines<br> - Alternative routes<br> <br> Operational Considerations:<br> - Aircraft types commonly used<br> - Typical delays and causes<br> - Seasonal performance metrics<br> - Airport congestion analysis<br> <br> <br> Pricing Information:<br> Base Fare: 49.00<br> Total Price: 234.18 EUR<br> <br> Detailed Flight Information:<br> <br> Flight Segment Analysis:<br> Carrier: B6 118<br> Equipment: E90<br> <br> Departure Details:<br> Airport: JFK<br> Terminal: 5<br> ...</code>	<code>Hotel Comprehensive Profile: TUNE HOTEL PADDINGTON<br> <br> Location Analysis:<br> City: LON<br> Precise Location: 51.5183, <br> -0.17069<br> Country: GB<br> <br> Property Details:<br> Chain: XN<br> Category: Standard Hotel<br> Last Updated: 2023-06-15T10:24:21<br> <br> Neighborhood Overview:<br> <br> Neighborhood Characteristics:<br> - Local atmosphere and vibe<br> - Safety and security assessment<br> - Proximity to business districts<br> - Entertainment and dining options<br> - Cultural attractions nearby<br> - Shopping facilities<br> - Green spaces and recreation<br> <br> Transportation Hub Analysis:<br> - Major transit stations<br> - Bus and tram routes<br> - Taxi availability<br> - Bike-sharing stations<br> <br> Local Life:<br> - Popular local venues<br> - Markets and shopping areas<br> - Cultural institutions<br> - Sports facilities<br> <br> <br> ...</code>	<code>0.297061319001456</code>
  <code>Local Transportation:<br> '''New York''' (known as "The Big Apple", "NYC," and often called "New York City") is a global center for media, entertainment, art, fashion, research, finance, and trade. The bustling, cosmopolitan heart of the 4th largest metropolis in the world and by far the most populous city in the United States, New York has long been a key entry point and a defining city for the nation.<br>From the Statue of Liberty in the harbor to the Empire State Building towering over the Manhattan skyline, from the tun<br> <br> Nearby Attractions:<br> '''New York''' (known as "The Big Apple", "NYC," and often called "New York City") is a global center for media, entertainment, art, fashion, research, finance, and trade. The bustling, cosmopolitan heart of the 4th largest metropolis in the world and by far the most populous city in the United States, New York has long been a key entry point and a defining city for the nation.<br>From the Statue of Liberty ...</code>	<code>Comprehensive Flight Analysis: LON to ROM<br> <br> Route Overview:<br> <br> Route Characteristics:<br> - Distance and typical duration<br> - Common connection points<br> - Seasonal weather impact<br> - Time zone considerations<br> <br> Market Analysis:<br> - Popular travel periods<br> - Price fluctuation patterns<br> - Competing airlines<br> - Alternative routes<br> <br> Operational Considerations:<br> - Aircraft types commonly used<br> - Typical delays and causes<br> - Seasonal performance metrics<br> - Airport congestion analysis<br> <br> <br> Pricing Information:<br> Base Fare: 80.00<br> Total Price: 119.32 EUR<br> <br> Detailed Flight Information:<br> <br> Flight Segment Analysis:<br> Carrier: VY 6225<br> Equipment: 320<br> <br> Departure Details:<br> Airport: LGW<br> Terminal: S<br> ...</code>	<code>0.12655311984152326</code>

• Loss: <code>MatryoshkaLoss</code> with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          384,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 10
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

</details>

Training Logs

Framework Versions

• Python: 3.13.1
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.6.0+cpu
• Accelerate: 0.26.0
• Datasets: 3.3.1
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->