osmosis-crossencoder-joint

OSMoSIS Cross-Encoder Joint Classifier

Five-class response sufficiency classifier using DeBERTa-v3 as a cross-encoder. Takes (objective, response) pairs as separate inputs with direct cross-attention between the two texts.

Performance

Per-Class F1 (Triage held-out, 374 samples)

Only 3 misclassifications out of 374 test samples.

Progression Across Approaches

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model = AutoModelForSequenceClassification.from_pretrained("KingTechnician/osmosis-crossencoder-joint")
tokenizer = AutoTokenizer.from_pretrained("KingTechnician/osmosis-crossencoder-joint")

labels = ["ADDR_DIRECT", "ADDR_PARTIAL", "NOADDR_ON", "NOADDR_TANGENTIAL", "NOADDR_OFF"]

objective = "What causes rain?"
response = "Rain forms when water vapor in the atmosphere condenses into droplets."

inputs = tokenizer(objective, response, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    logits = model(**inputs).logits
    prediction = logits.argmax(dim=-1).item()

print(f"Prediction: {labels[prediction]}")
# Output: Prediction: ADDR_DIRECT

Architecture

This is a cross-encoder, not a bi-encoder. The model processes [CLS] objective [SEP] response [SEP] as a single input, allowing full self-attention between objective and response tokens. This is critical for response sufficiency classification because the judgment depends on token-level alignment between what was asked and what was answered.

Training

• Base model: MoritzLaurer/deberta-v3-base-zeroshot-v2.0 (NLI-pretrained)
• Data: Yahoo Answers OSMoSIS (24,890) + Triage synthetic (1,492) = 26,382 pairs
• Epochs: 10, LR: 2e-5 with linear warmup (10%)
• Loss: Cross-entropy with class weights [0.57, 1.31, 0.52, 2.41, 6.84]
• Final train accuracy: 98.4%

Labels