nexus-risk-scorer

NEXUS Risk Scorer

risk_scorer.pkl — a Platt-calibrated logistic regression that maps a 9-dimensional plan-feature vector to a per-plan risk score ρ(P) ∈ [0, 1]. It is the learned component of the NEXUS runtime safety monitor for tool-using LLM agents.

NEXUS combines this scorer with a deterministic rule set (V_R) and an argument-level inspector (V_A) to produce one of four graded interventions per plan: ALLOW · BLOCK · CONFIRM · REVISE.

• Paper / repo: github.com/eliashossain001/nexus
• Companion datasets: EliasHossain/nexus-stress · EliasHossain/nexus-ipi · EliasHossain/nexus-synthetic · EliasHossain/nexus-multistep

---

TL;DR

---

Intervention policy

The scorer feeds the formal intervention policy Π:

Π(P) =
  BLOCK    if ∃ v ∈ V(P) : sev(v) = CRIT
  BLOCK    if ρ(P) ≥ τ_b ∧ |V(P)| ≥ 1
  CONFIRM  if ρ(P) ≥ τ_c ∨ ∃ v : sev(v) = HIGH
  REVISE   if ∃ v : sev(v) = MED
  ALLOW    otherwise

V(P) = V_R(P) ∪ V_A(P) is the union of rule and argument-inspector violations; sev(·) ∈ {CRIT, HIGH, MED, LOW}. Thresholds are selected by minimising

L(Π) = E[λ_s · u_s + λ_o · u_o + λ_c · c(Π(P))]

over a 5×5 grid of (λ_s, λ_o) weights. The point (0.75, 0.70) is loss-optimal uniformly across the grid on the synthetic split.

---

9-D plan feature vector

The scorer consumes per-plan features summarising side effects, sensitivity, permissions, network reach, budget, and structural shape:

An earlier 10-D variant included a redundant normalised-plan-length feature; it was retired and the scorer retrained — all numbers below reflect the 9-D model.

---

Quick start

from huggingface_hub import hf_hub_download
import pickle, numpy as np

ckpt_path = hf_hub_download(
    repo_id="EliasHossain/nexus-risk-scorer",
    filename="risk_scorer.pkl",
)
ckpt = pickle.load(open(ckpt_path, "rb"))
model, scaler = ckpt["model"], ckpt["scaler"]

# 9-D feature vector for some plan P
features = np.array([[3, 1, 1, 0, 3, 2, 4.0, 0.08, 0]])
rho = model.predict_proba(scaler.transform(features))[0, 1]

tau_b, tau_c = 0.75, 0.70
decision = (
    "BLOCK"    if rho >= tau_b else
    "CONFIRM"  if rho >= tau_c else
    "ALLOW"
)
print(f"ρ = {rho:.3f}  →  {decision}")

For the full policy (rules + argument inspector + scorer), install the package and use EnhancedSafetyMonitor:

git clone https://github.com/eliashossain001/nexus.git && cd nexus
pip install -e .

from runtime_safety.monitors.enhanced.monitor import EnhancedSafetyMonitor
monitor = EnhancedSafetyMonitor.from_default_checkpoint()
intervention, reasons = monitor.decide(plan)

---

Performance

Bootstrap CIs use 1000 resamples with seed=0.

---

Calibration

Calibration set: 60 held-out plans, seed=7. Reliability diagram is reproducible from scripts/eval/eval_calibration.py in the source repo.

---

Files

risk_scorer.pkl    # {'model': LogisticRegression, 'scaler': StandardScaler}
chunks.pkl         # RAG knowledge-base chunks (Nexora KB)
faiss.index        # FAISS index over the chunks

chunks.pkl + faiss.index are the retrieval cache used by the demo agent (scripts/demo/demo_*.py). They are not required to run the scorer itself but ship together so the full agent + monitor stack is reproducible.

---

Intended use & limitations

Intended for: research on runtime safety for tool-using LLM agents; ablating rule-based vs. learned components of agent intervention policies.

Not intended for:

• standalone safety adjudication on out-of-distribution agent stacks without re-calibration;
• threat models where the harmful and benign variants of a request use identical tool calls (AgentHarm-style), where the scorer collapses to the rule-only baseline by construction.

Known limitation — middle-severity coverage gap. On rule-blind adversarial plans (NEXUS-Stress), Π predicts only ALLOW or BLOCK and never CONFIRM / REVISE. We disclose this as a deployment-relevant gap; future rule-set extensions should target medium-severity scope-tighten and disambiguation patterns.

---

Reproducibility

All experiments are deterministic. Train/test split uses seed=42, train/calibration seed=7, benchmark generators and bootstrap seed=0. Reproducible from a fresh checkout in under 10 minutes on CPU.

Citation

@inproceedings{hossain2026nexus,
  title     = {NEXUS: Structured Runtime Safety for Tool-Using LLM Agents},
  author    = {Hossain, Elias and Nipu, Md Mehedi Hasan and Ornee, Tasfia Nuzhat and Rana, Rajib and Yousefi, Niloofar},
  booktitle = {ACL Rolling Review},
  year      = {2026}
}

License

Apache-2.0.