minilm-tekla-attr-embed-v1

MiniLM Tekla Attribute Embeddings (v1)

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L12-v2 for matching Tekla Structures attributes with natural language queries. It enables semantic search and attribute matching by mapping natural language descriptions to Tekla attribute names.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L12-v2 <!-- at revision 936af83a2ecce5fe87a09109ff5cbcefe073173a -->
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset: Synthetically generated Tekla attribute-query pairs (35,590 samples)
• Language: English
• License: GNU General Public License v3.0

Model Overview

This model is specifically designed for Tekla Structures attribute matching. It converts natural language queries (e.g., "bolt name from catalog") into embeddings that can be matched against Tekla attribute names (e.g., "BOLT_FULL_NAME").

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

# Example: Tekla Attribute Matching
from sentence_transformers import SentenceTransformer, util

# Load the model
model = SentenceTransformer("teknovizier/minilm-tekla-attr-embed-v1")

# Tekla attributes to match against
attributes = ["TOP_LEVEL", "PRODUCT_NAME", "POUR_PHASE", "BOLT_FULL_NAME"]

# User queries (natural language)
queries = ["object top level", "product name", "pour phase", "bolt name from catalog"]

# Get embeddings
attr_embeddings = model.encode(attributes)
query_embeddings = model.encode(queries)

# Calculate similarities and find best matches
similarities = util.cos_sim(query_embeddings, attr_embeddings)
for i, query in enumerate(queries):
    best_match_idx = similarities[i].argmax().item()
    print(f"'{query}' → '{attributes[best_match_idx]}' (score: {similarities[i][best_match_idx]:.3f})")

# Output:
# 'object top level' → 'TOP_LEVEL' (score: 0.822)
# 'product name' → 'PRODUCT_NAME' (score: 0.879)
# 'pour phase' → 'POUR_PHASE' (score: 0.953)
# 'bolt name from catalog' → 'BOLT_FULL_NAME' (score: 0.670)

Text Normalization

For optimal performance, text should be normalized before encoding:

1. Convert to lowercase
2. Replace underscores and hyphens with spaces
3. Remove non-alphanumeric characters

Example: "BOLT_NAME_FROM_CATALOG" → "bolt name from catalog"

The model is trained on normalized text, so applying this normalization to both queries and attribute names will yield the best matching results.

Limitations

Synthetic Training Data

This model is trained on synthetically generated data created using Claude Opus. While the synthetic data generation process includes quality filtering and validation, the model's performance on real-world user queries may differ from its performance on synthetic training examples.

Domain Specificity

The model is specifically fine-tuned for Tekla Structures attribute matching and may not generalize well to:

• Other BIM or engineering software
• General natural language understanding tasks
• Attribute names from other software systems

Text Normalization Dependency

Optimal performance requires consistent text normalization (lowercasing, underscore/hyphen replacement). Queries that don't follow this normalization pattern may yield suboptimal matches.

Training Details

Training Dataset

Tekla Attribute Matching Dataset

• Size: 35,590 training samples (query-attribute pairs)
• Data Source: Synthetically generated using Claude Opus (Anthropic's AI model)
• Attribute Source: Tekla Structures 2025 attribute names
• Columns: <code>query</code> (natural language) and <code>attribute</code> (Tekla attribute name)
• Sample Pairs:

  query	attribute
  <code>bolt name from catalog</code>	<code>BOLT_FULL_NAME</code>
  <code>pour phase</code>	<code>POUR_PHASE</code>
  <code>object top level</code>	<code>TOP_LEVEL</code>
  <code>weld cross section area above line</code>	<code>WELD_CROSSSECTION_AREA1</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• multi_dataset_batch_sampler: round_robin

Training Logs

Framework Versions

• Python: 3.13.12
• Sentence Transformers: 5.2.2
• Transformers: 5.1.0
• PyTorch: 2.12.0.dev20260308+cu128
• Accelerate: 1.13.0
• Datasets: 4.6.1
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Model Card Authors

• Model Development: teknovizier
• GitHub: teknovizier
• HuggingFace: teknovizier