laal-embedding-v0

laal-embedding-v0

laal-embedding-v0 is a Sentence-Transformers embedding model fine-tuned from intfloat/multilingual-e5-large-instruct for improved retrieval-oriented semantic search, with a focus on Korean fire-safety and legal-domain text.

• Base model: intfloat/multilingual-e5-large-instruct
• Embedding dimension: 1024
• Similarity function: cosine
• Max tokens: 512
• Architecture: XLM-RoBERTa (24 layers)
• HF repo: https://huggingface.co/jjp97/laal-embedding-v0

⚠️ Important This model uses fixed instruction prefixes defined in config_sentence_transformers.json. Always pass raw text to encode(). Do NOT manually prepend instruction strings.

---

Prompting (Important)

This model applies different fixed prefixes depending on the input type.

Query prefix

Instruct: Given a web search query, retrieve relevant passages that answer the query.
Query:

Passage prefix

title: none
text:

These prefixes are automatically applied by Sentence-Transformers via config_sentence_transformers.json.

Correct usage ✅

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("jjp97/laal-embedding-v1")

q_emb = model.encode_query("화재 시 대피 방법")
p_emb = model.encode_document("화재가 발생하면 즉시 119에 신고하고 안전한 경로로 대피해야 한다.")

Incorrect usage ❌ (double-prefixing)

# Do NOT do this
q = "Instruct: Given a web search query, retrieve relevant passages...\nQuery: 화재 시 대피 방법"
emb = model.encode(q)

---

Training

Objective

• Contrastive learning (InfoNCE)

• In-batch negatives

• Temperature (tau): 0.05

• Regularization: GOR (spread-out loss)

  • gor_lambda = 0.001
  • gor_max_samples = 64

Data

• Training examples: 43,983

• Format: (query, positive passage)

• Hard negatives: enabled

  • max_hn_per_example_train = 2

Training data consists of domain-specific Korean fire-safety and legal documents (private / curated dataset).

Hyperparameters (summary)

• Batch size: 512
• Epochs: 3
• Learning rate: 1e-5
• Warmup ratio: 0.1
• Approx. total steps: 255

---

Model Architecture

This model follows the standard Sentence-Transformers pipeline:

1. Transformer: XLM-RoBERTa (24 layers, hidden size 1024)
2. Pooling: mean pooling
3. Normalization: L2 normalization

---

Intended Use

• Retrieval and semantic search (RAG pipelines)
• Domain-specific QA (fire safety, legal text)
• Embedding-based similarity and clustering (best performance in retrieval-style settings)

---

Evaluation

Sanity check

Query–passage cosine similarity shows reasonable separation between relevant and irrelevant passages.

MTEB

This model is intended for evaluation on the MTEB leaderboard. When reporting results, please specify:

• model name: jjp97/laal-embedding-v0
• exact revision (commit hash)
• benchmark suite used

---

Limitations

• Performance may degrade if instruction prefixes are manually added (double-prefixing).
• Fine-tuned primarily for retrieval; performance on classification/STS tasks may vary.
• Domain bias toward Korean fire-safety / legal text.

---

License

• Released under Apache-2.0, following the base model license.

---

Acknowledgements

• Base model: Multilingual E5, Liang Wang et al., Multilingual E5 Text Embeddings, arXiv:2402.05672
• Sentence-Transformers library

---

Citation

If you use this model, please cite:

@misc{laal_embedding_v0_2025,
  title = {laal-embedding-v0},
  author = {Park, Jeongjae},
  year = {2025},
  howpublished = {Hugging Face model card},
}