Agent Testing overview

Agent Testing turns important text-agent behavior into repeatable checks. A saved test can be attached to an Agent, run against an explicit configuration snapshot, and reviewed with the same definition and evidence later. Use it before publishing a risky prompt, changing a Tool or Workflow, or shifting production traffic.

Agent Testing and Test Chat are different#

Test Chat is an interactive conversation for exploring one draft or published branch tip. It is useful while writing, but it does not create a reusable evaluator or a batch result. Agent Testing stores a versioned definition, runs it asynchronously, and records pass, fail, interrupted, usage, latency, transcript, and Tool evidence for each run.

Both surfaces are text-only. Agent Testing does not place phone calls, send Telegram messages, or reproduce a channel transport. A run evaluates the selected Agent configuration inside the workspace.

Choose a test type#

• Simulation runs a multi-turn scenario with one or more success conditions.
• Next Reply evaluates the Agent's next response from a supplied conversation history.
• Tool Invocation checks a Tool call or Workflow transition and can verify that it did not happen.

Start with Simulation for an outcome across a conversation, Next Reply for response quality at one decision point, or Tool Invocation and mocks for structured action evidence.

Production workflow#

Create and review tests in the workspace library, attach the relevant set to the Agent or Workflow node, choose Main Draft or Live explicitly, then run. Treat failures as release evidence: inspect the individual run before editing the Agent, because a failed evaluator, quota stop, and unmet condition require different recovery.

Saved revisions and invocation snapshots make old results explainable after the current test or Agent changes. Past results are not rewritten. See Attach and run tests and Past runs and results.