toolcall-verifier

ToolCallVerifier - Unauthorized Tool Call Detection

<div align="center">

Stage 2 of Two-Stage LLM Agent Defense Pipeline

</div>

---

🎯 What This Model Does

ToolCallVerifier is a ModernBERT-based token classifier that detects unauthorized tool calls in LLM agent systems. It performs token-level classification on tool call JSON to identify malicious arguments that may have been injected through prompt injection attacks.

---

🚨 Attack Categories Covered

🔗 Integration with FunctionCallSentinel

This model is Stage 2 of a two-stage defense pipeline:

┌─────────────────┐     ┌──────────────────────┐     ┌─────────────────┐
│   User Prompt   │────▶│ ToolCallSentinel │────▶│   LLM + Tools   │
│                 │     │      (Stage 1)       │     │                 │
└─────────────────┘     └──────────────────────┘     └────────┬────────┘
                                                              │
                               ┌──────────────────────────────▼──────────────────────────┐
                               │           ToolCallVerifier (This Model)                 │
                               │   Token-level verification before tool execution        │
                               └─────────────────────────────────────────────────────────┘

---

🎯 Intended Use

Primary Use Cases

• LLM Agent Security: Verify tool calls before execution
• Prompt Injection Defense: Detect unauthorized actions from injected prompts
• API Gateway Protection: Filter malicious tool calls at infrastructure level

Out of Scope

• General text classification
• Non-tool-calling scenarios
• Languages other than English

📜 License

Apache 2.0