OBLIQ-IR-3B-no-distill

OBLIQ-IR-3B-no-distill

This is an ablation model, not the main system. It is OBLIQ-IR trained without the stylometric kNN-graph distillation — the OBLIQ-IR (no distillation) row in every table of OBLIQ-IR: Training a Dense Retriever for Oblique Queries (EMNLP 2026).

👉 For the actual system, use DataScience-UIBK/OBLIQ-IR-3B.

What exactly is removed

The name is short, so to be precise about it: this model is fully trained. It sees the complete per-mechanism synthetic mixture — 149,361 rows across all five tasks, with BM25 hard negatives, the same backbone, the same LoRA configuration, the same optimiser, the same schedule.

The only difference is that the 5,000 authorship kNN-graph pairs are absent from the writing subset. Nothing else changes. It is an ablation of one ingredient, not an untrained or partially-trained model.

Why it exists

It isolates what the kNN-graph distillation contributes. Removing those 5,000 rows costs Writing-Style 0.115 NDCG@10 while barely moving the other three tasks — which is the paper's central claim, and the reason the distillation is applied to writing only.

Distillation buys a large gain on the one task whose latent attribute has no topical footprint, and costs under 0.01 on the three where a topical lens already works.

Usage

Identical to the main model:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("DataScience-UIBK/OBLIQ-IR-3B-no-distill", trust_remote_code=True)
q = model.encode(["query: "   + "your query"])
d = model.encode(["passage: " + "your document"])
scores = model.similarity(q, d)

The query: / passage: prefixes are required. Do not prepend a task instruction — the model was neither trained nor evaluated with one.

A note on precision

These are merged full weights: W + BA·scaling folded into bf16. bf16 carries only about three significant digits, so folding is slightly lossy and this repository does not reproduce its source adapter bit-for-bit. Measured on this checkpoint:

Small enough to be irrelevant for use, large enough to move the third decimal — Writing displays as .095 here against the paper's .096. The table above quotes the paper's adapter-form numbers.

Links

• 🤖 Main model — OBLIQ-IR-3B
• 🧩 Main adapter — OBLIQ-IR-3B-LoRA
• 📊 Data and runs — OBLIQ-IR-Data
• 💻 Code — github.com/DataScienceUIBK/obliq-ir
• 📄 Paper — OBLIQ-IR: Training a Dense Retriever for Oblique Queries

Licence

CC BY-NC 4.0. Redistributes weights derived from nvidia/llama-nv-embed-reasoning-3b, which NVIDIA releases for non-commercial / research use (LICENSE_nvidia_base_model.txt, NOTICE.txt). Built with Llama — the base derives from meta-llama/Llama-3.2-3B and the Llama 3.2 Community License also applies.

Citation

@inproceedings{abdalla2026obliqir,
  title     = {{OBLIQ-IR}: Training a Dense Retriever for Oblique Queries},
  author    = {Abdalla, Mahmoud and Abdallah, Abdelrahman and Sedek, Shaimaa and Jatowt, Adam},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
  year      = {2026}
}