sec-bert-causal-classifier_s2

sec-bert-causal-classifier_s2

A DeBERTa-v2 sequence classifier that detects causal vs. non-causal text segments in financial reports (e.g. SEC filings, MD&A sections). It is designed to act as a fast, recall-oriented pre-filter in a larger cause-and-effect extraction pipeline: it flags paragraphs likely to contain causal language so that only those segments are forwarded to a downstream LLM agent for structured (cause, effect) tuple extraction — dramatically reducing LLM API cost and latency.

Model Details

Model Description

• Developed by: Imad Eddine HAJJANE
• Model type: Binary sequence classification (causal / non-causal)
• Language(s): English (financial domain)
• Finetuned from: microsoft/deberta-v2-xlarge
• Architecture: DebertaV2ForSequenceClassification — 24 layers, hidden size 1024, 16 attention heads, max sequence length 512

This model is part of a broader financial causal-relation extraction project. Causal detection in financial text is non-trivial: causal sentences are a minority class (~10–20%), causality is often implicit, and documents contain multi-hop causal chains and counterfactual expressions.

Uses

Direct Use

Classify a financial text segment as causal (contains a cause-effect relationship) or non-causal. Intended primarily as a high-recall filtering stage so downstream extraction only processes relevant paragraphs.

Downstream Use

Front-end filter in a LangGraph pipeline: report → split into paragraphs → this classifier flags causal paragraphs → an LLM agent (e.g. Claude) extracts structured (cause, effect) tuples from the flagged paragraphs only.

Out-of-Scope Use

Not intended for non-financial domains, non-English text, or as a standalone source of truth for causal relationships without downstream verification. The reasons:

• Non-financial domains — the model was fine-tuned on financial reports (SEC filings, MD&A) where causal language follows domain-specific patterns (e.g. "revenue declined due to..."). On other domains it sees vocabulary and causal phrasings absent from training, so accuracy is not guaranteed.
• Non-English text — both the DeBERTa-v2 base and the fine-tuning data are English-only, so the model has no learned representation for other languages.
• Not a standalone source of truth — it is deliberately built as a high-recall pre-filter, accepting false positives to avoid missing true causal segments. It also only flags whether a segment is causal, not what the cause and effect are. The actual (cause, effect) extraction and verification are handled by the downstream LLM agent, so the classifier's output should always be confirmed there rather than trusted on its own.

How to Get Started

Recommended: custom handler (handler.py)

This repository ships a custom EndpointHandler (handler.py) that loads the model as a single-logit classifier, reads the tuned decision threshold directly from config.json (threshold_F1), and returns labeled predictions. This is the recommended path — it works out of the box on Hugging Face Inference Endpoints and requires no manual thresholding.

from handler import EndpointHandler

handler = EndpointHandler(path="Imad17700/sec-bert-causal-classifier_s2")

result = handler({
    "inputs": [
        "Revenue declined 8% in Q3 due to lower demand in the European market.",
        "The board will meet next Tuesday.",
    ]
})
# [
#   {"text": "...", "label": "CAUSAL",     "score": 0.91, "p_causal": 0.91, "threshold": 0.35},
#   {"text": "...", "label": "NON-CAUSAL", "score": 0.04, "p_causal": 0.04, "threshold": 0.35},
# ]

The handler accepts either a single string or a list of strings, batches them, applies sigmoid to the single logit, and labels each as CAUSAL / NON-CAUSAL using the threshold stored in the config — so the operating point travels with the model and never has to be hardcoded by callers.

Manual / transformers usage

If you prefer to load the model directly, read the threshold from the config rather than hardcoding it:

import torch
from transformers import AutoConfig, AutoTokenizer, AutoModelForSequenceClassification

model_id = "Imad17700/sec-bert-causal-classifier_s2"
config = AutoConfig.from_pretrained(model_id)
config.num_labels = 1
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(
    model_id, config=config, ignore_mismatched_sizes=True
)
model.eval()

# Tuned operating point loaded straight from config.json
threshold = float(getattr(model.config, "threshold_F1", 0.5))

text = "Revenue declined 8% in Q3 due to lower demand in the European market."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    prob_causal = torch.sigmoid(model(**inputs).logits.squeeze(-1))[0].item()

label = "CAUSAL" if prob_causal >= threshold else "NON-CAUSAL"
print(label, round(prob_causal, 4))

Note: this checkpoint is a single-logit classifier (num_labels = 1) and ships a generic id2label in config.json. Treat the positive class as causal, and load the operating threshold from config.json (threshold_F1, or threshold_precision / threshold_recall) rather than assuming 0.5.

Training Details

Training Data

Training data combines public financial causal benchmarks (e.g. FinCausal 2020/2023) with a custom-generated dataset. The custom data was produced through a LangGraph orchestration pipeline using LLMs to generate and label both real and synthetic causal / non-causal financial segments, expanding coverage of implicit causality, multi-hop chains, and domain-specific jargon and helping to mitigate the natural class imbalance.

Training Procedure

Single-label (binary) classification fine-tuning of DeBERTa-v2 using parameter-efficient fine-tuning (PEFT) with LoRA adapters. Each iteration trains LoRA adapters on top of the base model rather than updating all weights; if the adapter-equipped checkpoint passes evaluation, the adapter is merged into the base model and the decision threshold is updated from that iteration's test phase. Class imbalance is addressed through a combination of the synthetic-augmentation strategy above and operating-threshold tuning (rather than relying on a default 0.5 cutoff).

Agentic training pipeline

Training and iteration are orchestrated as a LangGraph state machine driven by an LLM agent equipped with the Tavily search tool. The agent automates the full experiment loop — planning a run, building/augmenting data, validating it, training LoRA adapters (PEFT), testing, and conditionally merging the adapters into the base model before deciding whether to iterate again or stop.

Several nodes — PLAN, VALIDATE, and TEST — each emit a report that is passed forward to the next node as feedback. Most importantly, the TEST report is fed back to PLAN, which the agent uses (together with Tavily search) to re-extract real data and regenerate synthetic data targeted at the weaknesses surfaced in testing, closing the improvement loop.

The loop works as follows:

1. PLAN — the LLM agent (with Tavily search) defines the next iteration (data mix, hyperparameters, target operating point) and emits a plan report. When fed the previous TEST report, it re-extracts real data and regenerates synthetic data to address the reported weaknesses.
2. WORKER — build / generate / augment the training data for the iteration (the LLM-driven real + synthetic data step), guided by the plan report.
3. VALIDATE — check the prepared data and configuration and emit a validation report, then route into TRAIN (or back to PLAN).
4. TRAIN — fine-tune LoRA adapters (PEFT) on top of the DeBERTa-v2 base model using the validated data.
5. TEST — evaluate the adapter-equipped checkpoint, tune thresholds, and emit a test report that is passed back to PLAN for the next iteration. The result decides the next edge:
   • MERGE — if the model with the adapters passes the test, the LoRA adapters are merged into the base model, and the decision threshold is updated based on this iteration's test phase, then it flows to PREP_NEXT_ITERATION.
   • NO_MERGE — otherwise, the adapters are discarded (not merged), the threshold is left unchanged, and it goes straight to PREP_NEXT_ITERATION.
6. PREP_NEXT_ITERATION — consolidate state and apply the stop criteria:
   • FAIL — criteria not met, so loop back to PLAN for another iteration.
   • PASS — criteria met, so terminate at __end__.

This closed-loop design lets the model improve over successive iterations with automated data generation, validation gating, and conditional LoRA-adapter merging — only adapters that pass the test are merged into the base model, and the operating threshold in config.json is refreshed from that same test phase. The final published weights are therefore a fully merged DeBERTa-v2 checkpoint (no separate adapter required at inference) carrying the latest tuned threshold.

Training Hyperparameters

• Fine-tuning method: LoRA adapters via PEFT, merged into the base model on passing iterations
• Max sequence length: 512
• Hidden dropout / attention dropout: 0.1
• Precision: float32

Evaluation

Thresholds & Operating Points

Instead of a static 0.5 decision boundary, this model uses probability thresholds tuned on a validation set and stored in config.json. Three operating points are provided so the threshold can be matched to the downstream use case:

The included handler.py reads threshold_F1 from config.json at load time, so the tuned operating point ships with the model and is applied automatically at inference — no need to hardcode a threshold in client code. To switch operating points, point the handler at threshold_precision or threshold_recall, or override the config value.

The evaluation reporting captures threshold, accuracy, F1, precision, recall, and per-class (causal / non-causal) precision, recall, and F1. Because this model is used as a cost-saving pre-filter, a recall-focused operating point is preferred in production: missing a true causal segment is more costly than passing a few false positives to the downstream LLM, which can discard them.

Bias, Risks, and Limitations

• Trained on English financial text; performance will degrade on other domains or languages.
• Partly trained on LLM-generated synthetic data, which may introduce stylistic artifacts not present in real filings; the agentic feedback loop (test report → plan) mitigates this by re-extracting real data and regenerating synthetic data to target observed weaknesses.
• As a binary filter, it does not localize the exact cause/effect spans — that is delegated to the downstream extraction stage.
• Threshold choice materially changes behavior; always select the operating point that matches your precision/recall needs.

Citation

If you use this model, please cite:

@misc{hajjane_sec_causal_classifier_2026,
  title  = {sec-bert-causal-classifier_s2: A DeBERTa-v2 causal/non-causal classifier for financial text},
  author = {Hajjane, Imad Eddine},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Imad17700/sec-bert-causal-classifier_s2}}
}

Model Card Contact

Imad Eddine HAJJANE