ats-mpnet-model

SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-mpnet-base-v2 <!-- at revision e8c3b32edf5434bc2275fc9bab85f82640a19130 -->
• Maximum Sequence Length: 384 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'MPNetModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Unity Development. Game Development. Mobile Game Development. Game Design. Game Art. 2D Game Art. 3D Game Art. 3D Modeling. Level Design. Game Prototype. Game UI/UX Design. Augmented Reality. Virtual Reality. Animation. VFX. Game Documentation. Game Balance. Analytics Integration. Monetization Strategy. User Acquisition. Unity. Unity Game Development. Creates full-cycle 2D and 3D mobile games from concept to launch and support.. Game Art and UI/UX. Designs 2D/3D art assets, animations, VFX, and user interfaces for games.. Game Design and Monetization. Builds GDDs, wireframes, balance systems, analytics integration, and monetization features.. Senior Game Development Expert | Mind Studios. Works on full-cycle development of 2D and 3D mobile games, from idea to launch and support (September 2016 - Present).',
    'Machine Learning Engineer\n\n        Requirements:\n        Python, Machine Learning, Deep Learning,\n        NLP, TensorFlow, PyTorch, SQL,\n        Data Science, Artificial Intelligence,\n        Generative AI, Transformers.',
    'Machine Learning Engineer\n\n        Requirements:\n        Python, Machine Learning, Deep Learning,\n        NLP, TensorFlow, PyTorch, SQL,\n        Data Science, Artificial Intelligence,\n        Generative AI, Transformers.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.0610, 0.0610],
#         [0.0610, 1.0000, 1.0000],
#         [0.0610, 1.0000, 1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 2,953 training samples
• Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  modality	text	text
  details	<ul><li>min: 91 tokens</li><li>mean: 269.21 tokens</li><li>max: 384 tokens</li></ul>	<ul><li>min: 34 tokens</li><li>mean: 39.33 tokens</li><li>max: 42 tokens</li></ul>	<ul><li>min: 0.04</li><li>mean: 0.21</li><li>max: 0.82</li></ul>

• Samples:

  sentence_0	sentence_1	label
  <code>Program management. Cybersecurity. Risk assessment. Vulnerability management. Business continuity. Standardization. Policy and coordination. Web security. Adversary emulation. Adversary emulation services. Enterprise CIS security frameworks. Business continuity planning tools. Web Presence Security Program. Managed a NATO web presence security program to improve cybersecurity posture and implement adversary emulation services.. Enterprise CIS Security Initiatives. Led enterprise-wide CIS security coherence and vulnerability management activities to strengthen NATO security and resilience.. NATO Business Continuity Project. Coordinated business continuity activities to support NATO operational resilience during transitions.. Program Manager - NATO OCIO. Manage cybersecurity programs focused on ICT coherence, enterprise CIS security, vulnerability management, and adversary emulation (March 2022 - Present).. Business Continuity Project Coordinator - NATO. Coordinated business continuity p...</code>	<code>Cybersecurity Engineer<br><br> Requirements:<br> Network Security, Ethical Hacking,<br> Penetration Testing,<br> Vulnerability Assessment,<br> Python, Linux, Wireshark,<br> Nmap, SIEM,<br> Incident Response,<br> Digital Forensics.</code>	<code>0.7298</code>
  <code>Cloud Platform Engineering. DevOps. Site Reliability Engineering. Infrastructure as Code. CI/CD. Kubernetes. Docker. Cloud Migration. Observability. Security and Compliance. Terraform. Platform Architecture. Database / Data Platform. EKS. ECS. Lambda. VPC. IAM. RDS. S3. CloudWatch. CloudTrail. GuardDuty. API Gateway. AKS. Data Factory. Functions. Key Vault. Entra ID. Private Link. Azure Monitor. Terraform. Helm. Kustomize. Ansible. ArgoCD. Flux. GitHub Actions. GitLab CI. Azure DevOps. Jenkins. Prometheus. Grafana. ELK. Loki. OpenTelemetry. PagerDuty. Vault. OPA. Kyverno. Trivy. Snowflake. ADLS Gen2. Power BI. Python. Bash. YAML. HCL. On-prem to AWS Migration. Led phased zero-downtime migration from on-prem to AWS with multi-AZ EKS clusters, segmented VPCs, and modular Terraform reducing infrastructure drift by ~90%.. Cloud-native Platform with GitOps. Designed cloud-native platform using Kubernetes, ArgoCD, hardened CI/CD with image scanning and SAST, full observability, Vault secrets...</code>	<code>Machine Learning Engineer<br><br> Requirements:<br> Python, Machine Learning, Deep Learning,<br> NLP, TensorFlow, PyTorch, SQL,<br> Data Science, Artificial Intelligence,<br> Generative AI, Transformers.</code>	<code>0.1524</code>
  <code>Cyber resilience. Information security management. ISO 27001. Risk management. Security assessments. Gap analysis. CISO-as-a-Service. Project management. Customer success. Training and awareness. ISO/IEC 27001. NIST CSF. OneTrust (inferred from GRC tooling experience). M365. BSI IT-Grundschutz. ISMS Implementation and Audit. Led ISMS build, audits, security assessments and gap analyses aligned with ISO/IEC 27001 and NIST CSF to improve organisational security posture.. Regulatory Readiness for NIS2 & CRA. Advised clients on compliance and translated regulatory requirements (NIS2, Cyber Resilience Act, FINMA) into practical measures and governance.. M365 Rollout and OKR Adoption. Managed M365 rollout projects and introduced OKR agile management practices while coordinating project risks and stakeholder engagement.. Cyber Security Consultant - InfoGuard AG. Consulting on cyber security, ISMS implementation, audits, risk analyses and regulatory compliance including NIS2 and Cyber Resilien...</code>	<code>Cloud Engineer<br><br> Requirements:<br> AWS, Azure, Google Cloud,<br> Docker, Kubernetes,<br> Terraform, Linux,<br> DevOps, CI/CD,<br> Cloud Infrastructure,<br> Cloud Security.</code>	<code>0.16920000000000002</code>

• Loss: <code>CosineSimilarityLoss</code> with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss",
      "cos_score_transformation": "torch.nn.modules.linear.Identity"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• per_device_train_batch_size: 16
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 16
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

Training Time

• Training: 13.4 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.6.0
• Transformers: 5.13.1
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->