AI agent evaluation: measuring agents that plan, call tools, and act.

What is AI agent evaluation?

AI agent evaluation is the practice of measuring how well an AI agent completes tasks in a dynamic environment: planning steps, calling tools, handling state, and reaching a goal. Unlike a static model test, it scores an end-to-end system on both the outcome it reaches and the trajectory it takes to get there, using a mix of code checks, model judges, and human review.

The vocabulary that has settled in 2026 is useful: an evaluation runs a set of tasks, each over one or more trials, scored by graders, producing transcripts (the trajectories). You measure the outcome (did it reach the goal) and the trajectory (how it got there) as separate things, because an agent can get the right answer the wrong way.

How is evaluating an AI agent different from evaluating an LLM?

Evaluating an LLM tests a model on a mostly static input and output. Evaluating an agent tests a system that takes many steps, calls external tools, keeps state across turns, and acts in an environment that changes as it goes. The core difficulty is non-determinism and path variability: the same task can be solved by many different, valid sequences, so you cannot just match one expected output.

Agent evaluation builds on the base metrics and judge techniques of language-model evaluation, so the mechanics of scoring text, LLM-as-a-judge, and its biases carry over. For those foundations see our guide to LLM evaluation; this page focuses on what is specific to agents.

What do you measure when evaluating an AI agent?

Agent evaluation measures a stack of things: task completion (did it reach the goal), tool-call correctness (right tools, right arguments), trajectory quality (was the path sound and efficient), reliability across repeated runs, safety and policy adherence, and cost and latency. Each metric applies at a level, the individual step, the whole trajectory, or the full session, and is scored by code, an LLM judge, or a human.

To make that concrete: take a support agent that must look up an order and issue the right refund. One golden task fixes the request and the correct end state, then scores several axes at once, whether it called the lookup and refund tools with valid arguments (tool-call correctness), in a sensible order without redundant calls (trajectory and step efficiency), reached the correct end state (task success), and succeeded on all eight of eight repeated runs (pass^8 reliability) rather than just once.

How do you evaluate tool calling (tool-call correctness)?

You evaluate tool calling by checking, at each step, whether a tool call was warranted, whether the right tool was chosen, and whether its arguments were correct. Those three checks, scored against the tools and arguments a correct run would have used, are what tool-call correctness measures.

Tool selection is scored as precision and recall against the expected calls, matched by tool name: precision is the share of calls that were correct, recall the share of expected calls it made. Argument correctness is stricter, the call must be schema-valid and its parameter values must match, often measured as the share of correct key-value pairs. Function-calling benchmarks like Berkeley's BFCL score exactly this, using abstract syntax tree comparison rather than string matching.

What is trajectory evaluation and how is it scored?

Trajectory evaluation scores the path an agent took, the ordered sequence of reasoning, tool calls, and observations, not just its final answer. It is scored by comparing the actual step sequence to an expected one under a match mode, and by measuring efficiency against the optimal path.

The standard match modes, as defined by Google's Agent Development Kit and Vertex AI evaluation, are exact match (the trajectory must match step for step), in-order match (the expected steps must appear in order, extra steps allowed), and any-order match (the expected steps must all appear, order aside); LangSmith offers analogous modes under different names. You pick the strictness the task warrants. Trajectory scoring matters because outcome-only evaluation hides failures: an agent can reach the right end state through a path that is wasteful, non-reproducible, or unsafe, and only trajectory-level scoring surfaces that.

How do you evaluate multi-turn and long-horizon agents?

Multi-turn agents are evaluated with simulated users, one model plays a persona and interacts with the agent over many turns, plus reliability metrics across repeated runs. Many agent failures emerge only after the first turn, as context drift, forgotten constraints, or loops, so single-turn scoring misses them.

Reliability is where two metrics matter. Pass@k is the chance that at least one of k attempts succeeds, a measure of capability. Pass^k is the chance that all k attempts succeed, a measure of reliability, and it is the harder, more honest bar for production. Sierra's tau-bench popularized pass^k precisely because agents that look strong at pass@1 often collapse when you require them to succeed every time. In the original 2024 benchmark even state-of-the-art agents solved under half of its tasks, and pass^k reliability keeps falling as k grows.

How do you measure agent cost, latency, and efficiency?

You measure cost and efficiency as tokens consumed per task, number of tool calls and steps versus the optimal, wall-clock completion time, and iteration count. An agent that reaches the right answer after twenty redundant tool calls is not production-ready, so efficiency is a first-class metric, not an afterthought.

These operational numbers are also where an agent's economics live. Two agents with the same task success rate can differ tenfold in cost per completed task, and that gap, not the success rate, is often what decides whether the agent is worth deploying at scale.

What are the common AI agent failure modes?

The common failure modes cluster into a small taxonomy: wrong tool selected, malformed or wrong arguments, hallucinated or non-existent tools, non-termination and loops, context drift over long interactions, and cost or latency blowups. Naming the failure class for each bad run is what makes agent evaluation actionable rather than just a pass/fail number.

• Wrong tool: the agent picks a plausible but incorrect tool for the step.
• Bad arguments: the right tool, but invalid or wrong parameter values.
• Hallucinated tool: the agent invents a tool or capability that does not exist.
• Loops / non-termination: the agent repeats steps or never stops.
• Context drift: over many turns it forgets constraints or the original goal.
• Cost / latency blowup: it reaches the goal but through far too many steps.

Which benchmarks are used to evaluate AI agents?

The widely cited agent benchmarks include tau-bench (tool-agent-user tasks), SWE-bench (real software issues), WebArena (web navigation), AgentBench (cross-environment), GAIA (general assistant tasks), OSWorld (computer use), and BFCL (function calling). They are useful for tracking the frontier and comparing models, but they measure generic capability on someone else's tasks, not how an agent performs on your workflows.

The scale of the gap is worth internalizing. On GAIA, human respondents score around 92% while an early GPT-4-with-plugins system scored roughly 15%. Frontier numbers have climbed since, but the lesson holds: a benchmark tells you how a model does on a public test, never how your agent does on the tasks your business actually runs.

What tools and frameworks evaluate AI agents?

A growing set of frameworks handle the mechanics of agent evaluation and tracing, including LangSmith, DeepEval, Braintrust, Arize Phoenix, MLflow, and the model providers' own evaluation suites. They differ on what matters for agents: open-source versus commercial, trajectory scoring versus final-output only, offline versus online, and, most important for an enterprise, whether they can be self-hosted inside your own perimeter. We compare the categories, and where an owned capability fits, in AI evaluation tools, compared.

How do you evaluate an AI agent on your own data and workflows?

You evaluate an agent on your own data by building golden tasks from your real workflows and past production failures, running them in your own environment with your tools and policies, scoring both outcome and trajectory against what your business considers correct, and keeping the whole evaluation invisible to the model vendors you are judging. Public benchmarks cannot do this; only an owned evaluation can.

In practice that is a handful of steps, run as a continuous loop rather than a one-time test:

1. Mine real failures into a golden set of 20 to 50 representative tasks from your workflows.
2. Run in your environment with your real tools, data, and policies, inside your perimeter.
3. Score outcome and trajectory against your own definition of a correct, safe, efficient run.
4. Re-run on every change of model, prompt, or tool, and keep it invisible to the vendors you evaluate.

This is where agent evaluation meets the harder strategic point: the criteria that define a good run for your business are your intellectual property, and a metric your vendor can see is a metric your vendor will optimize for. We make that argument in full in what private AI evaluation is; the short version is that the evaluation deciding real money has to be owned by you and invisible to the vendors it judges.

Common questions about AI agent evaluation