Block Unsafe LangGraph Tool Calls Before They Run

The model chooses which tool to call and with what arguments. In this guide's calculator agent, add and multiply are the intended tools, yet the model can also call divide. Run the agent with no check and divide goes through: nothing stopped it. For a calculator that is harmless. Swap divide for issue_refund or delete_user and it would run just the same.

In a LangGraph agent, a tool call crosses two places. bind_tools tells the model which tools exist, and ToolNode runs the tool the model picks. The edge between the two is where a request becomes an action, so that is where governed_tools puts the check.

What happens on one tool call

1. Intercept. The wrapped tool receives the call from ToolNode. Your tool's code has not run.
2. Ask. The wrapper sends the tool name and its arguments, with the agent id, to the AgentTrust gateway. This is the pre-check.
3. Compare. The gateway checks the call against the agent's declared tools. A tool that is not declared, or an amount above its declared limit, scores Tool Trust 0, a hard stop.
4. Record. The gateway writes an event for the decision, allowed or refused.
5. Run or refuse. An allowed call runs, and its result goes back to the gateway for a post-check, a second event. A refused call never runs. The model receives Blocked by AgentTrust as the tool's result.

Why before the tool and not after

A check that runs after a step can hold back a result. It cannot undo an action that already happened, such as a refund that was sent. That is why the AutoGen, OpenAI Agents, Claude and MCP adapters are described as post-step checks, and why governed_tools is the one to use when the action itself must not run. AgentTrustNode is not this check.

What the wrapped list must cover

The check covers only the tools you wrap. Use the wrapped list in both bind_tools and ToolNode. A tool you pass unwrapped is not checked: with the same 500 refund, an unwrapped tool ran it. The manifest matters too. An agent with no declared tools is not restricted, so every tool is allowed.

Layer	What it sees	Can it stop the action?
bind_tools	Which tools the model may ask for	No. It only shapes what the model sees.
governed_tools on ToolNode	The tool name and arguments, before the tool runs	Yes. A refused call never runs.
Post-step adapters	The result, after the step ran	No. They can hold back a result.

The same four steps, applied to LangGraph's calculator agent. You run it three ways and watch a tool call get refused. About 10 minutes. Each step has one thing to do and one thing you should see.

Before you start. Python 3.10 or later, and Ollama for a chat model that runs on your machine. Steps 1 to 4 need nothing else. Steps 5 to 7 use an AgentTrust console account.

Create the project

Make a folder with its own Python environment and install what the program needs:

mkdir agentrust-quickstart && cd agentrust-quickstart
python -m venv .venv && source .venv/bin/activate     # on Windows: .venv\Scripts\activate
pip install "agentrust-py[langgraph,embedded]" "langchain>=1.0" langchain-ollama
ollama pull llama3.1

Create a file named langgraph_quickstart.py in that folder, paste the program below and save. It is the calculator agent from LangGraph's own quickstart. The tools (add, multiply, divide) are at the top, the agent is a model plus a tool node in a loop, AgentTrust is one line (governed_tools, step 3), and the three ways to run it are at the bottom.

The example agent. The model asks for a tool; governed_tools asks the AgentTrust gateway before the tool runs; the gateway records every check as an event.

Here is what happens when the agent is asked to add, then to divide. The tool names come from the program you are about to create.

1. The model asks for a tool. llm_call decides it needs add (or divide) and passes the request to tool_node.
2. The governed tool asks AgentTrust first. The wrapped tool sends the tool name and its arguments to the gateway. This is the pre-check, and your tool's code has not run.
3. The gateway decides and records it. It compares the call with the agent's declared tools: add is declared, divide is not. Either way it writes an event.
4. Allowed: the tool runs. Refused: it never does. add(3, 4) runs and its result goes back for a post-check, a second event. divide becomes a Blocked by AgentTrust result instead.
5. The model gets a result either way. It reads the answer or the refusal, replies, and the graph finishes.

So an allowed call leaves two events (before and after it ran) and a refused call leaves one. With --console they appear under Events in step 7; with --local the gateway lives inside the program and forgets them when it exits.

See the whole program

langgraph_quickstart.py

"""
langgraph_quickstart.py — a LangGraph agent whose tool calls AgentTrust checks.

The agent is the calculator from LangGraph's own quickstart: a chat model with
add, multiply and divide tools, in a loop that runs until the model answers
without asking for a tool.

AgentTrust is one line (marked "AgentTrust" below): the tools are wrapped with
governed_tools. Every call is checked before it runs, and its result after. A
tool the agent may not use is refused before its code runs; the model is told
why and the graph still finishes. The agent may use add and multiply, not divide.

RUN IT (in the folder that holds this file)
    pip install "agentrust-py[langgraph,embedded]" "langchain>=1.0" langchain-ollama
    ollama pull llama3.1                                  # a free model with tool calling

    python langgraph_quickstart.py --without-agentrust    # AgentTrust off: every tool runs
    python langgraph_quickstart.py --local                # a gateway starts inside this program; no account
    python langgraph_quickstart.py --console              # your AgentTrust console keeps every decision

  --console needs three settings, from the agent you create in the console:
      export AGENTRUST_GATEWAY_URL=https://console.agent-trust.tech
      export AGENTRUST_KEY=at_live_...
      export AGENTRUST_AGENT_ID=...

ANOTHER MODEL
    QUICKSTART_MODEL=anthropic:claude-sonnet-4-6 (pip install langchain-anthropic,
    export ANTHROPIC_API_KEY). Any init_chat_model string for a model that
    supports tool calling works.
"""
from __future__ import annotations

import argparse
import operator
import os
import sys
import tempfile
from typing import Annotated, Literal

from langchain.chat_models import init_chat_model
from langchain.messages import AnyMessage, HumanMessage, SystemMessage
from langchain.tools import tool
from langgraph.graph import END, START, StateGraph
from typing_extensions import TypedDict

from agentrust_sdk.adapters.langgraph import governed_tools

AGENT_ID = os.environ.get("AGENTRUST_AGENT_ID", "calculator-agent")
MODEL = os.environ.get("QUICKSTART_MODEL", "ollama:llama3.1")

# The tools this agent may call. With --local they are set here; with --console
# you declare the same list on the agent in the console. divide is left out on purpose.
MANIFEST = {"declared_tools": [{"name": "add"}, {"name": "multiply"}]}

# Step 1: Define tools
@tool
def multiply(a: int, b: int) -> int:
    """Multiply `a` and `b`.

    Args:
        a: First int
        b: Second int
    """
    return a * b

@tool
def add(a: int, b: int) -> int:
    """Adds `a` and `b`.

    Args:
        a: First int
        b: Second int
    """
    return a + b

@tool
def divide(a: int, b: int) -> float:
    """Divide `a` and `b`.

    Args:
        a: First int
        b: Second int
    """
    return a / b

# AgentTrust: every call to these tools is checked before it runs and after it
# returns. Use this same list for bind_tools and for the tool node.
tools = governed_tools([add, multiply, divide], agent_id=AGENT_ID)
tools_by_name = {t.name: t for t in tools}

# Step 2: Define state
class MessagesState(TypedDict):
    messages: Annotated[list[AnyMessage], operator.add]
    llm_calls: int

# Step 4: Define tool node
def tool_node(state: MessagesState) -> dict:
    """Performs the tool calls the model asked for."""
    result = []
    for tool_call in state["messages"][-1].tool_calls:
        t = tools_by_name[tool_call["name"]]
        # Invoking with the whole tool call returns a ToolMessage. A call
        # AgentTrust refuses comes back as ToolMessage(status="error").
        result.append(t.invoke(tool_call))
    return {"messages": result}

# Step 5: Define end logic
def should_continue(state: MessagesState) -> Literal["tool_node", END]:
    """Route to the tool node if the model made a tool call, else stop."""
    if state["messages"][-1].tool_calls:
        return "tool_node"
    return END

# Steps 3 and 6: Define the model node, then build and compile the agent
def build_agent(model):
    model_with_tools = model.bind_tools(tools)

    def llm_call(state: MessagesState) -> dict:
        """LLM decides whether to call a tool or not."""
        return {
            "messages": [
                model_with_tools.invoke(
                    [SystemMessage(content="You are a helpful assistant tasked with performing arithmetic on a set of inputs.")]
                    + state["messages"]
                )
            ],
            "llm_calls": state.get("llm_calls", 0) + 1,
        }

    agent_builder = StateGraph(MessagesState)
    agent_builder.add_node("llm_call", llm_call)
    agent_builder.add_node("tool_node", tool_node)
    agent_builder.add_edge(START, "llm_call")
    agent_builder.add_conditional_edges("llm_call", should_continue, ["tool_node", END])
    agent_builder.add_edge("tool_node", "llm_call")
    return agent_builder.compile()

def ask_agent(where: str) -> bool:
    """Ask the agent to add, then to divide. False if the model can't be reached."""
    print(f"AgentTrust: {where} · agent {AGENT_ID} · model {MODEL}")
    agent = build_agent(init_chat_model(MODEL, temperature=0))
    try:
        for request in ["Add 3 and 4.", "Divide 10 by 2."]:
            print(f"\n=== {request} ===")
            state = agent.invoke({"messages": [HumanMessage(content=request)]})
            for m in state["messages"]:
                m.pretty_print()
    except Exception as exc:  # noqa: BLE001 - any provider error ends the run
        print(f"\nThe model call failed ({MODEL}): {exc}\n"
              "Check the model provider is reachable (Ollama: `ollama serve` and `ollama pull llama3.1`).",
              file=sys.stderr)
        return False
    return True

# --without-agentrust: every tool runs. For comparison.
def run_without_agentrust() -> int:
    os.environ["AGENTRUST_ENABLED"] = "false"   # the SDK's off switch: wrapped tools pass every call through
    return 0 if ask_agent("off (every tool runs)") else 1

# --local: a gateway runs inside this program; no account needed.
def run_local() -> int:
    from agentrust_sdk.embedded import embed_gateway

    # SQLite in a temp dir, gone after the run. embed_gateway sets
    # AGENTRUST_GATEWAY_URL and AGENTRUST_KEY, which governed_tools reads.
    gateway = embed_gateway(db_path=os.path.join(tempfile.mkdtemp(), "gateway.db"))
    gateway.register_manifest(AGENT_ID, MANIFEST)
    try:
        ok = ask_agent(f"local gateway {gateway.url} (no account, nothing kept)")
    finally:
        gateway.stop()
    return 0 if ok else 1

# --console: the SDK sends every check to your console, which keeps the decisions.
def run_with_console() -> int:
    missing = [v for v in ("AGENTRUST_GATEWAY_URL", "AGENTRUST_KEY", "AGENTRUST_AGENT_ID") if not os.environ.get(v)]
    if missing:
        print(f"Set {', '.join(missing)} (from the agent you created in the console), or use --local.",
              file=sys.stderr)
        return 2
    console = os.environ["AGENTRUST_GATEWAY_URL"]
    ok = ask_agent(f"console {console}")
    print(f"\nOpen Events in the console ({console}) and filter by agent ID {AGENT_ID}.")
    return 0 if ok else 1

def main(argv: list[str] | None = None) -> int:
    parser = argparse.ArgumentParser(description="The LangGraph calculator agent, governed by AgentTrust.")
    where = parser.add_mutually_exclusive_group()
    where.add_argument("--local", action="store_true", help="run a gateway inside this program, no account (the default)")
    where.add_argument("--console", action="store_true", help="send every check to your AgentTrust console")
    where.add_argument("--without-agentrust", action="store_true", help="for comparison: AgentTrust off, every tool runs")
    args = parser.parse_args(argv)
    if args.without_agentrust:
        return run_without_agentrust()
    return run_with_console() if args.console else run_local()

if __name__ == "__main__":
    sys.exit(main())

Check that the file works:

python langgraph_quickstart.py --help

usage: langgraph_quickstart.py [-h]
                               [--local | --console | --without-agentrust]

If Python cannot find a package, the environment from the first command is not active. Run its second line again.

Run it without AgentTrust

First, see what the agent does on its own:

python langgraph_quickstart.py --without-agentrust

The agent is asked to add, then to divide. The run takes 20 to 40 seconds on a laptop, longer the first time while the model loads.

AgentTrust: off (every tool runs) · agent calculator-agent · model ollama:llama3.1
...
=== Divide 10 by 2. ===
...
Name: divide

5.0

The agent used divide and nothing stopped it. For a calculator that is harmless. Swap divide for issue_refund or delete_user and it would run just the same.

Where governed_tools sits in the LangGraph file

This is the only change AgentTrust needs, and it is already in your file. Nothing to edit. It sits right after the tools are defined and before the agent is built. The wrapped list is used in both places the agent uses its tools: for the model and for the tool node. On your own agent it looks like this:

Diff

+ from agentrust_sdk.adapters.langgraph import governed_tools

- tools = [add, multiply, divide]
+ tools = governed_tools([add, multiply, divide], agent_id=AGENT_ID)

  model_with_tools = model.bind_tools(tools)   # the wrapped list here
  graph.add_node("tools", ToolNode(tools))     # and here

Names, descriptions and argument types stay the same, so the model sees what it saw before. Allowed calls run and their results are checked. Refused calls never run.

Run it locally

Run the same agent with AgentTrust on:

python langgraph_quickstart.py --local

The program starts a small AgentTrust gateway inside itself and allows only the tools in MANIFEST: add and multiply. divide is not on the list. No account needed.

AgentTrust: local gateway http://127.0.0.1:8765 (no account, nothing kept) · agent calculator-agent · model ollama:llama3.1
...
=== Add 3 and 4. ===
...
Name: add

7
...
=== Divide 10 by 2. ===
...
Name: divide

Blocked by AgentTrust: Tool-trust hard gate: action 'divide' is not in this agent's declared_tools (envelope 43c1d418-…)

add ran. divide was refused before its code ran, and the model was told why, so the agent still finished. Nothing is kept after the program exits.

Create the agent in your console

To keep every decision and manage the agent from the console, register it there:

1. Open Agents, select New agent, name it langgraph-quickstart, and select Create.
2. Copy the API Key and the Agent ID before you select Done. The key starts with at_live_ and is shown only once. Never paste it in a chat, a ticket or a commit.
3. Select the agent. Under Manifest & fingerprint, select Declare. Under Declared tools, select + Add tool twice, type add and multiply, and select Save manifest.

That list is the same one MANIFEST holds in the program.

Run it against the console

Give the program the three values from step 5, then run it with --console:

.env

AGENTRUST_GATEWAY_URL=https://console.agent-trust.tech   # your console's address
AGENTRUST_KEY=at_live_...                                # the API key from step 5
AGENTRUST_AGENT_ID=langgraph-quickstart                  # the Agent ID from step 5

# Optional
AGENTRUST_FAILURE_MODE=closed     # refuse tool calls when AgentTrust can't be reached (default: open)

export AGENTRUST_GATEWAY_URL=https://console.agent-trust.tech   # your console's address
export AGENTRUST_KEY=at_live_...                                # the API key
export AGENTRUST_AGENT_ID=langgraph-quickstart                  # the Agent ID

python langgraph_quickstart.py --console

AgentTrust: console http://127.0.0.1:18000 · agent langgraph-quickstart · model ollama:llama3.1
...
Name: divide

Blocked by AgentTrust: Tool-trust hard gate: tool_trust: action 'divide' is not in this agent's declared_tools (envelope d05dc011-…)

Open Events in the console (http://127.0.0.1:18000) and filter by agent ID langgraph-quickstart.

Same result as step 4. This real run used a self-hosted AgentTrust gateway, so the first line shows its address; yours shows your console's.

See the decisions in the console

Open Events in the console and type your Agent ID into Filter by Agent ID.

Events. One row per decision: add approved before and after it ran, and divide blocked.

Select the Block row. Its Overview shows the scores behind the decision:

divide, refused. Tool Trust is 0 because divide is not declared. It is a Hard stop: refused even though the agent is in Alert mode.

Compare it with the approved add event:

add, approved. Every check passed and confidence is 100.

You are done. Your agent's tool calls are now checked, refused when they should not run, and recorded.

Wrap the list once. A tool you pass unwrapped is not checked.

Put @harness on plain Python tools in Stop Risky AI Agent Tool Calls in Python with @harness. Limit one CrewAI agent in Limit Which Tools Each CrewAI Agent Can Call. The full walkthrough is in the LangGraph integration docs.