bert-base-uncased-industrialtech

SentenceTransformer based on google-bert/bert-base-uncased

This is a sentence-transformers model finetuned from google-bert/bert-base-uncased on the wikipedia_subsets dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google-bert/bert-base-uncased <!-- at revision 86b5e0934494bd15c9632b12f734a8a67f723594 -->
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • wikipedia_subsets
• Language: en

<!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("UmarAzam/bert-base-uncased-industrialtech")
# Run inference
sentences = [
    'version of spaced was introduced beginning the 97th vehicle of 6th batch also introduced an of heavy ballistic Leopard on increased armour protection While Leopard to the Leopard 2A5 the covering the modules is modules . New armour modules armour cover the frontal arc of the turret . have distinctive and protection both penetrators and charges The side skirts incorporate improved protection . A 25 the danger of injuries in case armour penetration The Leopard 2A7 the generation and belly armour providing against mines and IEDs . Leopard 2A7 fitted for mounting armour modules protection systems against . For urban combat, the Leopard 2 can be with different of modular armour Leopard 2A4M Leopard 2 Peace) the mount modules composite along the flanks turret and hull, while slat armour can be adapted at vehicle The modules, which depending on the warhead can penetrate of armour The 2A6M CAN increases rocket-propelled including slat armour . Additional armour packages been developed by a number different companies IBD developed upgrades Advanced (AMAP) armour the latter used on Singaporean and Leopard tanks . RUAG has developed armour upgrade composite . first the 2013 . The Leopard and 2A6M add an additional protection for, which increases mines and IEDs . 22, the German Defence to Trophy, an active protection system of . 17 be fitted the with integration planned be in 2023 . Armour protection estimates Estimated levels of for range from 590 to 690 mm the turret RHAe the and lower front hull on Leopard 2A4, to mm RHAe turret 620 mm RHAe on',
    " version of spaced multilayer armour was introduced beginning with the 97th vehicle of the 6th production batch. The same batch also introduced an improved type of heavy ballistic skirts.\n\nThe Leopard 2A5 upgrade focused on increased armour protection. While upgrading a Leopard 2 tank to the Leopard 2A5 configuration, the roof covering the armour modules is cut open and new armour modules are inserted. New additional armour modules made of laminated armour cover the frontal arc of the turret. They have a distinctive arrowhead shape and improve protection against both kinetic penetrators and shaped charges. The side skirts also incorporate improved armour protection. A 25\xa0mm-thick spall liner reduces the danger of crew injuries in case of armour penetration.\n\nThe Leopard 2A7 features the latest generation of passive armour and belly armour providing protection against mines and IEDs. The Leopard 2A7 is fitted with adapters for mounting additional armour modules or protection systems against RPGs.\n\nFor urban combat, the Leopard 2 can be fitted with different packages of modular armour. The Leopard 2A4M CAN, Leopard 2 PSO (Peace Support Operations) and the Leopard 2A7 can mount thick modules of composite armour along the flanks of the turret and hull, while slat armour can be adapted at the vehicle's rear. The armour modules provide protection against the RPG-7, which depending on the warhead can penetrate between  and  of steel armour. The Leopard 2A6M CAN increases protection against rocket-propelled grenades (RPGs) by including additional slat armour.\n\nAdditional armour packages have been developed by a number of different companies. IBD Deisenroth has developed upgrades with MEXAS and Advanced Modular Armor Protection (AMAP) composite armour, the latter is being used on Singaporean and Indonesian Leopard 2 tanks. RUAG has developed an armour upgrade utilizing their SidePRO-ATR composite armour. This upgrade was first presented on the IAV 2013.\n\nThe Leopard 2A4M and 2A6M add an additional mine protection plate for the belly, which increases protection against mines and IEDs.\n\nOn 22 February 2021, the German Defence Ministry agreed to acquire Trophy, an active protection system of Israeli design. 17 German Army tanks will be fitted with the system, with integration planned to be completed in 2023.\n\nArmour protection estimates\nEstimated levels of protection for the Leopard 2 range from 590 to 690\xa0mm RHAe on the turret, 600\xa0mm RHAe on the glacis and lower front hull on the Leopard 2A4, to 920–940\xa0mm RHAe on the turret, 620\xa0mm RHAe on the",
    ", produced by George Haggerty, made by Kai Productions\n 28 December Incredible Evidence,  an Equinox Special about the limits of DNA profiling. Directed by Hilary Lawson, made by TVF\n\n1995\n 9 January Beyond Love, an Equinox Special about autoerotic asphyxia, which killed over 50 people in 1994; and due to the deeply, and distasteful, unconventional content of the programme, it was shown at 10pm; at the John Hopkins Sexual Disorders Clinic at the Johns Hopkins Bloomberg School of Public Health in Baltimore in Maryland, where chromosomal abnormality was found by Fred Berlin, often Klinefelter syndrome; Dr Raymond Goodman of Hope Hospital in Salford, now of the Institute of Brain, Behaviour and Mental Health at the University of Manchester, and why 90% of paraphiliacs were male; Peter Fenwick (neuropsychologist) of the Institute of Psychiatry, Psychology and Neuroscience, and how sexual arousal is centred in the limbic system; Gene Abel of the Behavioral Medicine Institute of Atlanta; William Marshall of the Queen's University at Kingston; Jeffrey Weeks (sociologist) at London South Bank University; John Bancroft (sexologist)  of the MRC Reproductive Biology Unit in Edinburgh; Stephen Hucker of Queen's University, Ontario; John Money of Johns Hopkins Hospital; forensic psychologist Ronald Langevin. Narrated by Dame Jenni Murray, directed by Peter Boyd Maclean, produced by Simon Andreae, made by Optomen Television\n 27 August  The Real X-Files: America's Psychic Spies, an Equinox Special about a former American military unit that conducted remote viewing, where operatives could see backwards and forwards in time; Admiral Stansfield Turner, Director from 1977 to 1981 of the CIA; Major-General Ed Thompson; Colonel John B. Alexander of the United States Army Intelligence and Security Command; Hal Puthoff, of SRI International in California; remote viewer Ingo Swann and the subsequent Stargate Project, at Fort Meade in Maryland; Keith Harary, who worked with Russell Targ. Narrated by Jim Schnabel, produced by Alex Graham, directed by Bill Eagles, made by Wall to Wall Television\n 3 September Cybersecrecy, the mathematician Fred Piper of the Information Security Group; the UK gave out Enigma machines to Commonwealth countries for secret telecommunications, without telling these countries that the UK could read every message; Phil Zimmermann, inventor of the PGP encryption algorithm; Simon Davies (privacy advocate); when at MIT in 1976, Whitfield Diffie found how to",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9618, 0.5859],
#         [0.9618, 1.0000, 0.5862],
#         [0.5859, 0.5862, 1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Semantic Similarity

• Datasets: sts-dev and sts-test
• Evaluated with <code>EmbeddingSimilarityEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

wikipedia_subsets

• Dataset: wikipedia_subsets at 72f5c2f
• Size: 81,516 training samples
• Columns: <code>text</code>
• Approximate statistics based on the first 1000 samples:

  	text
  type	string
  details	<ul><li>min: 512 tokens</li><li>mean: 512.0 tokens</li><li>max: 512 tokens</li></ul>

• Samples:

  text
  <code> Highway 82 where motorists enter the city's outskirts. The legal speed limit drops in a short space from 55 mph to 30 mph, leading to some drivers who are not alert to be caught. The minimum fine for exceeding the posted speed limit even by 1 mph is $146.<br><br>Initially, Illinois used photo enforcement for construction zones only. There was legislation on the books to expand that throughout the state. However, Chicago has expanded its red light camera program and is planning to put speed cameras in school zones. Some suburbs (e.g. Alsip) already have cameras at various intersections.<br><br>Some U.S. states that formerly allowed red-light enforcement cameras but not speed limit enforcement cameras ('photo radar'), have now approved, or are considering, the implementation of speed limit enforcement cameras. The Maryland legislature approved such a program in January 2006. In 2005, 2006, 2008 and 2009 the California legislature considered, but did not pass, bills to implement speed limit enforce...</code>
  <code> in many sectors of business including stock market trading systems, mobile devices, internet operations, fraud detection, the transportation industry, and governmental intelligence gathering.<br><br>The vast amount of information available about events is sometimes referred to as the event cloud.<br><br>Conceptual description<br><br>Among thousands of incoming events, a monitoring system may for instance receive the following three from the same source:<br><br> church bells ringing.<br> the appearance of a man in a tuxedo with a woman in a flowing white gown.<br> rice flying through the air.<br><br>From these events the monitoring system may infer a complex event: a wedding. CEP as a technique helps discover complex events by analyzing and correlating other events: the bells, the man and woman in wedding attire and the rice flying through the air.<br><br>CEP relies on a number of techniques, including:<br><br> Event-pattern detection<br> Event abstraction<br> Event filtering<br> Event aggregation and transformation <br> Modeling event hierarch...</code>
  <code>ating wheel that allows scientists to select between short, medium, and longer wavelengths when making observations using the MRS mode,” said NASA in a press statement.<br><br>Commissioning and testing<br>On 12 January 2022, while still in transit, mirror alignment began. The primary mirror segments and secondary mirror were moved away from their protective launch positions. This took about 10 days, because the 132 actuator motors are designed to fine-tune the mirror positions at microscopic accuracy (10 nanometer increments) and must each move over 1.2 million increments (12.5 mm) during initial alignment.<br><br>Mirror alignment requires each of the 18 mirror segments, and the secondary mirror, to be positioned to within 50 nanometers. NASA compares the required accuracy by analogy: "If the Webb primary mirror were the size of the United States, each [mirror] segment would be the size of Texas, and the team would need to line the height of those Texas-sized segments up with each other to an accurac...</code>

• Loss: <code>DenoisingAutoEncoderLoss</code>

Evaluation Dataset

wikipedia_subsets

• Dataset: wikipedia_subsets at 72f5c2f
• Size: 10,000 evaluation samples
• Columns: <code>text</code>
• Approximate statistics based on the first 1000 samples:

  	text
  type	string
  details	<ul><li>min: 512 tokens</li><li>mean: 512.0 tokens</li><li>max: 512 tokens</li></ul>

• Samples:

  text
  <code> prisoners of Stalin and Hitler, Frankfurt am Main; Berlin.<br> Wilfried Feldenkirchen: 1918–1945 Siemens, Munich 1995, Ulrike fire, Claus Füllberg-Stolberg, Sylvia Kempe: work at Ravensbrück concentration camp, in: Women in concentration camps. Bergen-Belsen. Ravensbrück, Bremen, 1994, pp. 55–69<br> Feldenkirchen, Wilfried (2000). Siemens: From Workshop to Global Player, Munich.<br> Feldenkirchen, Wilfried, and Eberhard Posner (2005). The Siemens Entrepreneurs: Continuity and Change, 1847–2005. Ten Portraits, Munich.<br> Greider, William (1997). One World, Ready or Not. Penguin Press. .<br> Sigrid Jacobeit: working at Siemens in Ravensbrück, in: Dietrich Eichholz (eds) War and economy. Studies on German economic history 1939–1945, Berlin 1999.<br> Ursula Krause-Schmitt: The path to the Siemens stock led past the crematorium, in: Information. German Resistance Study Group, Frankfurt / Main, 18 Jg, No. 37/38, Nov. 1993, pp. 38–46<br> MSS in the estate include Wanda Kiedrzy'nska, in: National Library of Pola...</code>
  <code> dates the beginning of behavioral modernity earlier to the Middle Paleolithic). This is characterized by the widespread observation of religious rites, artistic expression and the appearance of tools made for purely intellectual or artistic pursuits.<br> 49–30 ka: Ground stone tools – fragments of an axe in Australia date to 49–45 ka, more appear in Japan closer to 30 ka, and elsewhere closer to the Neolithic.<br> 47 ka: The oldest-known mines in the world are from Eswatini, and extracted hematite for the production of the red pigment ochre.<br> 45 ka: Shoes, as evidenced by changes in foot bone morphology in Eurasia. Bark sandals dated to 10 to 9 ka were found in Fort Rock Cave in the US state of Oregon in 1938. Oldest leather shoe (Areni-1 shoe), 5.5 ka.<br> 44–42 ka: Tally sticks (see Lebombo bone) in Eswatini<br> 43.7 ka: Cave painting in Indonesia<br> 37 ka: Mortar and pestle in Southwest Asia<br> 36 ka: Weaving – Indirect evidence from Moravia and Georgia. The earliest actual piece of woven cloth wa...</code>
  <code> on a prestressing. Prestressing means the intentional creation of permanent stresses in a structure for the purpose of improving its performance under various service conditions.<br><br>There are the following basic types of prestressing:<br> Pre-compression (mostly, with the own weight of a structure)<br> Pretensioning with high-strength embedded tendons<br> Post-tensioning with high-strength bonded or unbonded tendons<br>Today, the concept of prestressed structure is widely engaged in design of buildings, underground structures, TV towers, power stations, floating storage and offshore facilities, nuclear reactor vessels, and numerous kinds of bridge systems.<br><br>A beneficial idea of prestressing was, apparently, familiar to the ancient Roman architects; look, e.g., at the tall attic wall of Colosseum working as a stabilizing device for the wall piers beneath.<br><br>Steel structures <br><br>Steel structures are considered mostly earthquake resistant but some failures have occurred. A great number of welded steel mo...</code>

• Loss: <code>DenoisingAutoEncoderLoss</code>

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• learning_rate: 3e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 3e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary> </details>

Framework Versions

• Python: 3.10.18
• Sentence Transformers: 5.0.0
• Transformers: 4.53.2
• PyTorch: 2.7.1+cu126
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

DenoisingAutoEncoderLoss

@inproceedings{wang-2021-TSDAE,
    title = "TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding Learning",
    author = "Wang, Kexin and Reimers, Nils and Gurevych, Iryna",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    pages = "671--688",
    url = "https://arxiv.org/abs/2104.06979",
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->