TrustBench: Real-Time Pre-Execution Action Verification for AI Agents

Most agent safety work focuses on what happened after an action ran — analyzing logs, flagging violations in traces, or triggering rollback procedures. That approach has a fundamental flaw: by the time you know something went wrong, the harm is already done. Pre-execution verification flips the model, intercepting actions before they reach the environment and blocking or redirecting anything that fails a trust check.

The Problem with Post-Hoc Safety

Agents that interact with external systems — APIs, filesystems, databases, browsers — can cause side effects that are difficult or impossible to reverse. A tool call that deletes a record, sends an email, or commits a transaction is done the moment it executes. Detecting the violation afterward and logging it is useful for audits and debugging, but it doesn’t protect the system or the user.

Post-hoc evaluation is also structurally too slow for tight feedback loops. An agent running a multi-step task might produce dozens of tool calls per minute. Analyzing those calls asynchronously against a policy engine creates a growing backlog, and any remediation requires interrupting a workflow that has already advanced several steps past the offending action.

Pre-execution verification addresses this by sitting in the critical path: the agent generates an intended action, the action passes through a verification gate, and only cleared actions proceed to execution. The architectural challenge is doing this quickly enough that it doesn’t break the agent’s operational rhythm.

What a Trust Verification Gate Looks Like

A pre-execution verifier is a component that sits between the agent’s action selection layer and the tool executor. It receives a structured representation of the proposed action — typically the function name, arguments, and enough context about the agent’s current goal — and returns either a clearance decision or a rejection with a reason.

┌─────────────────────────────────────────────────────────┐
│                    Agent Runtime                        │
│                                                         │
│  ┌──────────┐    proposed    ┌──────────────────────┐   │
│  │   LLM    │ ─────action──▶ │  Trust Verifier Gate │   │
│  │(planner) │                │                      │   │
│  └──────────┘                │  ┌────────────────┐  │   │
│                              │  │ Policy Engine  │  │   │
│                              │  │  - scope rules │  │   │
│                              │  │  - risk model  │  │   │
│                              │  │  - context ctx │  │   │
│                              │  └────────────────┘  │   │
│                              │         │            │   │
│                              │    ALLOW / DENY      │   │
│                              └──────────┬───────────┘   │
│                                         │               │
│                         ┌───────────────┴──────────┐    │
│                         │                          │    │
│                   ┌─────▼──────┐           ┌───────▼──┐ │
│                   │    Tool    │           │  Reject  │ │
│                   │  Executor  │           │  + Log   │ │
│                   └────────────┘           └──────────┘ │
└─────────────────────────────────────────────────────────┘

The policy engine inside the verifier can use several mechanisms:

• Rule-based checks: explicit allowlists/denylists of tool names, argument patterns, or target resources
• Scope constraints: the agent was granted permission to read but not write, or to operate on a specific tenant’s data
• Risk scoring: a lightweight model or heuristic that estimates the reversibility and blast radius of the proposed action
• Contextual coherence: does this action make sense given the agent’s stated goal and recent history?

The first two are fast and deterministic. Risk scoring and coherence checking add accuracy at the cost of latency and complexity.

Engineering for Sub-200ms Verification

Keeping verification latency below 200ms — a threshold that keeps it imperceptible within a normal tool-call cycle — requires careful design. A few principles help:

Avoid calling the primary LLM for verification. Using the same model that generated the action to verify it is circular and expensive. Verification should use a smaller, faster model or a rule-based system.

Cache policy lookups. Most agents operate within a stable permission context. If the policy for a given agent identity and tool combination hasn’t changed, the result of checking read_file with a path under /allowed/ will be the same on every call. Caching these decisions with short TTLs dramatically reduces verification overhead.

Parallelize when possible. If an agent submits a batch of tool calls (common in parallel tool use), verification can fan out across all of them simultaneously rather than running sequentially.

import asyncio
from typing import NamedTuple

class ActionVerdict(NamedTuple):
    allowed: bool
    reason: str

async def verify_action(tool_name: str, args: dict, context: dict) -> ActionVerdict:
    # Fast path: check static allowlist first
    if tool_name in ALWAYS_ALLOWED_TOOLS:
        return ActionVerdict(allowed=True, reason="allowlist")
    
    if tool_name in ALWAYS_BLOCKED_TOOLS:
        return ActionVerdict(allowed=False, reason="blocklist")
    
    # Medium path: scope check
    if not scope_permits(tool_name, args, context["agent_permissions"]):
        return ActionVerdict(allowed=False, reason="out_of_scope")
    
    # Slow path: risk model (only if needed)
    risk_score = await compute_risk_score(tool_name, args, context)
    if risk_score > RISK_THRESHOLD:
        return ActionVerdict(allowed=False, reason=f"risk_score={risk_score:.2f}")
    
    return ActionVerdict(allowed=True, reason="passed")

async def verify_batch(actions: list[dict], context: dict) -> list[ActionVerdict]:
    return await asyncio.gather(*[
        verify_action(a["tool"], a["args"], context)
        for a in actions
    ])

Integrating Verification into an Agent Framework

The cleanest integration point is the tool executor itself — wrap it so that no tool can be called without passing through the verifier. This makes the safety guarantee unconditional regardless of how many different agent types or planners route through the same execution layer.

class VerifiedToolExecutor:
    def __init__(self, tool_registry, verifier, policy_context):
        self.tools = tool_registry
        self.verifier = verifier
        self.context = policy_context

    async def execute(self, tool_name: str, args: dict) -> dict:
        verdict = await self.verifier.verify_action(
            tool_name, args, self.context
        )
        
        if not verdict.allowed:
            # Surface structured rejection back to the planner
            raise ActionBlockedError(
                tool=tool_name,
                reason=verdict.reason,
                args=args
            )
        
        return await self.tools[tool_name].call(**args)

When an action is blocked, the rejection should be structured enough for the agent’s planner to understand why and attempt a recovery — either by choosing a different tool, asking for clarification, or escalating to a human. An opaque “denied” response leaves the agent unable to make a sensible next move.

What Gets Logged and Why

Every verification decision — allow and deny alike — should be logged with enough context to reconstruct why the verifier made the call. This serves multiple purposes: auditing whether the policy is calibrated correctly, identifying patterns of attempted policy violations, and providing ground truth for improving the risk model over time.

Useful fields in a verification log entry:

• Timestamp and agent session ID
• Tool name and sanitized argument summary (redact sensitive values)
• Verdict and the specific rule or risk score that drove it
• Whether the agent subsequently recovered or escalated
• End-to-end latency for the verification step

Pre-execution verification is most valuable when it’s tuned to be accurate enough that the teams operating agents trust it. A verifier that cries wolf on routine tool calls will be disabled; one that consistently catches genuine policy violations earns its place in the critical path.