ps-fullymasked-structbert-large

bayartsogt/structbert-large Fine-tuned on PiC Phrase Similarity (Angle 3 — with_rationale)

Cross-encoder model for the PiC (Phrase in Context) phrase-similarity binary classification task. Both sentences — plus an LLM-generated rationale — are fed together so the model can attend across them simultaneously.

Base model

bayartsogt/structbert-large

Input format

[CLS] sentence1_marked [SEP] sentence2_marked [SEP] rationale [SEP]

Target Phrase Marking

The target phrase is wrapped with <TGT>phrase</TGT> using substring matching against the raw sentence (whole-word boundaries preferred, case-insensitive fallback), since PS phrases can be multi-word and have no explicit position column in the source dataset.

Performance

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("Deehan1866/ps-fullymasked-structbert-large")
model = AutoModelForSequenceClassification.from_pretrained("Deehan1866/ps-fullymasked-structbert-large")

s1 = "In 1990, Petit accepted a full time overnight on <TGT>air position</TGT> at gospel radio station WYLD AM."
s2 = "In 1990, Petit accepted a full time overnight on <TGT>posture while jumping</TGT> at gospel radio station WYLD AM."
rationale = "The first phrase denotes a broadcasting role; the second describes a physical stance during a jump."

sep = tokenizer.sep_token
enc = tokenizer(s1, s2 + " " + sep + " " + rationale,
                return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    logits = model(**enc).logits
pred = torch.argmax(logits).item()
print("Similar" if pred == 1 else "Not similar")