lack-of-context-detector

Lack-of-Context Detector

The Lack-of-Context Detector is a 0.6B-parameter binary text classifier that estimates whether a question or instruction depends on context that is not present in the prompt. It is designed for filtering synthetic and mined instruction data before supervised fine-tuning or reinforcement learning.

The model is introduced in Data-Centric Post-Training for Financial Reasoning: Mining, Distillation, and Verifiable Learning.

Missing context is common in generated datasets: a question may refer to "the table above," an absent document, an undefined entity, an earlier conversation turn, or other evidence unavailable to the model. Training on such examples can encourage unsupported answers. This classifier provides a lightweight signal for detecting that failure mode at scale.

Model

The checkpoint adds a two-class sequence-classification head to Qwen/Qwen3-Embedding-0.6B. Inputs are user questions or instructions, truncated to at most 4,096 tokens. The semantic labels stored in config.json are:

The positive training target is MISSING_CONTEXT. For pipelines that prefer a self-containment score, use 1 - p(MISSING_CONTEXT).

Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="whoisjiji/lack-of-context-detector",
    tokenizer="whoisjiji/lack-of-context-detector",
)

prompts = [
    "Using the table above, calculate the company's operating margin.",
    "A bond pays a $50 annual coupon and costs $950. What is its current yield?",
]

scores = classifier(
    prompts,
    top_k=None,
    truncation=True,
    max_length=4096,
)

for prompt, result in zip(prompts, scores):
    probabilities = {item["label"]: item["score"] for item in result}
    p_missing = probabilities["MISSING_CONTEXT"]
    p_self_contained = 1.0 - p_missing
    print(prompt, {"missing_context": p_missing, "self_contained": p_self_contained})

The probabilities are selection signals rather than universal decision boundaries. Thresholds should be calibrated through manual review on the intended data source. A conservative filtering pipeline can require a high self-containment score; broader exploratory mining can retain uncertain examples for later review.

Limitations

• The model only sees the supplied prompt. It cannot determine whether relevant context exists elsewhere in an application or retrieval system.
• Some answerable questions rely on common knowledge, while some apparently complete questions still contain hidden ambiguity.
• Labels were distilled from a generative model and may reproduce its judgment biases.
• The model was trained on English financial and synthetic prompts and should not be assumed to generalize to other languages or domains.
• The score does not measure factual correctness, financial relevance, difficulty, or answer verifiability.
• This model is intended for dataset curation and research, not financial advice or automated high-stakes decisions.

Citation

@article{hayrapetyan2026datacentric,
  title   = {Data-Centric Post-Training for Financial Reasoning: Mining, Distillation, and Verifiable Learning},
  author  = {Hayrapetyan, Zhirayr and Kalmykov, Andrei and Kokosinskii, Denis and Stanishevskii, Dmitry and Zmitrovich, Dmitry},
  journal = {arXiv preprint arXiv:2609.10113},
  year    = {2026}
}

@misc{hayrapetyan2026lackofcontext,
  title  = {Lack-of-Context Detector},
  author = {Hayrapetyan, Zhirayr},
  year   = {2026},
  url    = {https://huggingface.co/whoisjiji/lack-of-context-detector}
}