TY-ecomm-asure-relevance

TY-ecomm-asure-relevance

<img src="trendyol_llm_asure.png" width="300" alt="Trendyol-LLM-Asure-Logo" />

A multilingual cross-encoder that scores how relevant a product is to a query, over the four ESCI classes. Distilled from Trendyol/Trendyol-LLM-Asure-12B.

Model Details

Labels

Intended Use

Use this model to classify product candidates for a user query by estimating whether each product is irrelevant, complementary, substitute, or relevant.

Limitations

The model can make incorrect relevance judgments and only uses the text fields provided at inference time. It does not see product images and should not be treated as having general world knowledge beyond the query-product text pair.

Input format

The first sequence is the query. The second is the product document: seven fields, one per line, separated by \n, using these exact English field names. They are the product attributes the model saw in training, so keep the names, the order and the separators.

title: {title}
category: {category}
brand: {brand}
gender: {gender}
age_group: {age_group}
marketplace_sellers: {seller1 | seller2 | ...}
attributes: {key1}: {value1}, {key2}: {value2}, ...

The tokenizer encodes the pair as <s> query </s></s> document </s>.

Sample input and output

Input:

query: "dönen mumluk"

title: truro mumluk naturel (11x14,5x5 cm)
category: ev ve mobilya / ev dekorasyon / şamdan mumluk
brand: bella maison
gender: unknown
age_group: unknown
marketplace_sellers: bella maison
attributes: renk: turuncu, materyal: ahşap, parça sayısı: 1, color detail: şeffaf

Output:

{'irrelevant': 0.0368, 'complementary': 0.0129, 'substitute': 0.9253, 'relevant': 0.025}

A rotating candle holder was searched for and a plain wooden one was found, so the product scores as substitute rather than relevant.

Installation

pip install torch "transformers>=4.57,<5"

transformers 5.x is not supported yet.

How to use

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "Trendyol/TY-ecomm-asure-relevance"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(model_id, trust_remote_code=True).eval()

query = "dönen mumluk"
document = "\n".join([
    "title: truro mumluk naturel (11x14,5x5 cm)",
    "category: ev ve mobilya / ev dekorasyon / şamdan mumluk",
    "brand: bella maison",
    "gender: unknown",
    "age_group: unknown",
    "marketplace_sellers: bella maison",
    "attributes: renk: turuncu, materyal: ahşap, parça sayısı: 1, color detail: şeffaf",
])

enc = tok(query, document, truncation=True, max_length=512, return_tensors="pt")
with torch.no_grad():
    probs = model(**enc).logits.softmax(-1)[0]
print({model.config.id2label[i]: round(p.item(), 4) for i, p in enumerate(probs)})

trust_remote_code=True is required: both the GTE encoder and the classification head are defined in this repository rather than in transformers.

Scoring many candidates for one query

queries = [query] * len(documents)
enc = tok(queries, documents, truncation=True, max_length=512,
          padding=True, return_tensors="pt")
with torch.no_grad():
    scores = model(**enc).logits.softmax(-1)[:, 3]  # P(relevant)

Training data

~1.5M query-product pairs in Turkish, Arabic, Romanian, English languages.

License

Apache-2.0

Citation

@misc{trendyol-ty-ecomm-asure-relevance,
  title  = {TY-ecomm-asure-relevance: Trendyol E-commerce Multilingual Query-Product Relevance Cross-Encoder},
  author = {Trendyol - LLM & Core NLP Team},
  year   = {2026},
  url    = {https://huggingface.co/Trendyol/TY-ecomm-asure-relevance}
}

Model Card Authors

Trendyol - LLM & Core NLP Team