Skip to content
OperadeDocs

Search documentation

Type to search documentation.

DashboardContact
Documentation
Copy page
Open raw Markdown

Create an agent test

Create a complete Simulation, Next Reply, or Tool Invocation test as an immutable initial revision, optionally attaching it to the caller's Agent draft atomically.

Endpoint

POST/api/v1/workspaces/{workspace_id}/agent-tests

Authentication

Operation classAuthenticationMinimum roleAPI key
customer-apiBearer API key or dashboard session cookie; dashboard mutations also require X-CX-Request: 1.memberAllowed

Parameters

NameInTypeRequiredDescription
workspace_idpathstringYes—

Request body

application/json request schema

Field pathTypeReferenceFormatNullableRequiredEnumDescription
$object——NoYes——

Choose exactly one variant using type.

Simulation variant (type = "simulation")

Field pathTypeReferenceFormatNullableRequiredEnumDescription
$object——NoYes——
attach_to_agent_idstring or null——YesNo——
chat_historyarray<HistoryEntry>——NoNo——
chat_history[]objectHistoryEntry—NoYes——
chat_history[].messagestring or null——YesNo——
chat_history[].rolestring——NoYes"user", "agent"—
chat_history[].tool_callsarray<ToolCall>——NoNo——
chat_history[].tool_calls[]objectToolCall—NoYes—Bagian stabil kontrak vendor; detail bertipe baru tetap dipertahankan.
chat_history[].tool_calls[].params_as_jsonstring——NoYes——
chat_history[].tool_calls[].request_idstring——NoYes——
chat_history[].tool_calls[].tool_detailsobject or null——YesNo——
chat_history[].tool_calls[].tool_has_been_calledboolean——NoYes——
chat_history[].tool_calls[].tool_namestring——NoYes——
chat_history[].tool_calls[].typestring or null——YesNo——
chat_history[].tool_resultsarray<ToolResult>——NoNo——
chat_history[].tool_results[]objectToolResult—NoYes——
chat_history[].tool_results[].is_blockedboolean——NoNo——
chat_history[].tool_results[].is_errorboolean——NoYes——
chat_history[].tool_results[].request_idstring——NoYes——
chat_history[].tool_results[].result_valuestring——NoYes——
chat_history[].tool_results[].tool_has_been_calledboolean——NoYes——
chat_history[].tool_results[].tool_latency_secsnumber——NoNo——
chat_history[].tool_results[].tool_namestring——NoYes——
chat_history[].tool_results[].typestring——NoYes——
conversation_initiation_sourcestring——NoNo"unknown", "telegram_integration"—
dynamic_variablesobject——NoNo——
evaluation_modelstring——NoNo——
from_conversation_metadataobject or nullFromConversationMetadata—YesNo——
from_conversation_metadata.conversation_idstring——NoYes——
from_conversation_metadata.event_seqinteger——NoYes——
from_conversation_metadata.original_agent_responsestring or null——YesNo——
from_conversation_metadata.system__conversation_idstring or null——YesNo——
from_conversation_metadata.system__time_utcstring—date-timeNoYes——
namestring——NoYes——
parent_folder_idstring or null——YesNo——
simulated_user_modelstring——NoNo——
simulation_max_turnsinteger——NoNo——
simulation_scenariostring——NoYes——
success_conditionsarray<string>——NoYes——
success_conditions[]string——NoYes——
tool_mock_configobjectToolMockConfig—NoNo——
tool_mock_config.fallback_strategystring——NoNo"raise_error", "call_real_tool"—
tool_mock_config.mocked_tool_idsarray<string>——NoNo——
tool_mock_config.mocked_tool_ids[]string——NoYes——
tool_mock_config.mocking_strategystring——NoNo"all", "selected", "none"—
tool_mock_overridesobject——NoNo——
typestring——NoYes"simulation"—

Next Reply variant (type = "llm")

Field pathTypeReferenceFormatNullableRequiredEnumDescription
$object——NoYes——
attach_to_agent_idstring or null——YesNo——
chat_historyarray<HistoryEntry>——NoNo——
chat_history[]objectHistoryEntry—NoYes——
chat_history[].messagestring or null——YesNo——
chat_history[].rolestring——NoYes"user", "agent"—
chat_history[].tool_callsarray<ToolCall>——NoNo——
chat_history[].tool_calls[]objectToolCall—NoYes—Bagian stabil kontrak vendor; detail bertipe baru tetap dipertahankan.
chat_history[].tool_calls[].params_as_jsonstring——NoYes——
chat_history[].tool_calls[].request_idstring——NoYes——
chat_history[].tool_calls[].tool_detailsobject or null——YesNo——
chat_history[].tool_calls[].tool_has_been_calledboolean——NoYes——
chat_history[].tool_calls[].tool_namestring——NoYes——
chat_history[].tool_calls[].typestring or null——YesNo——
chat_history[].tool_resultsarray<ToolResult>——NoNo——
chat_history[].tool_results[]objectToolResult—NoYes——
chat_history[].tool_results[].is_blockedboolean——NoNo——
chat_history[].tool_results[].is_errorboolean——NoYes——
chat_history[].tool_results[].request_idstring——NoYes——
chat_history[].tool_results[].result_valuestring——NoYes——
chat_history[].tool_results[].tool_has_been_calledboolean——NoYes——
chat_history[].tool_results[].tool_latency_secsnumber——NoNo——
chat_history[].tool_results[].tool_namestring——NoYes——
chat_history[].tool_results[].typestring——NoYes——
conversation_initiation_sourcestring——NoNo"unknown", "telegram_integration"—
dynamic_variablesobject——NoNo——
evaluation_modelstring——NoNo——
failure_examplesarray<string>——NoNo——
failure_examples[]string——NoYes——
from_conversation_metadataobject or nullFromConversationMetadata—YesNo——
from_conversation_metadata.conversation_idstring——NoYes——
from_conversation_metadata.event_seqinteger——NoYes——
from_conversation_metadata.original_agent_responsestring or null——YesNo——
from_conversation_metadata.system__conversation_idstring or null——YesNo——
from_conversation_metadata.system__time_utcstring—date-timeNoYes——
namestring——NoYes——
parent_folder_idstring or null——YesNo——
success_conditionstring——NoYes——
success_examplesarray<string>——NoNo——
success_examples[]string——NoYes——
typestring——NoYes"llm"—

Tool Invocation variant (type = "tool")

Field pathTypeReferenceFormatNullableRequiredEnumDescription
$object——NoYes——
attach_to_agent_idstring or null——YesNo——
chat_historyarray<HistoryEntry>——NoNo——
chat_history[]objectHistoryEntry—NoYes——
chat_history[].messagestring or null——YesNo——
chat_history[].rolestring——NoYes"user", "agent"—
chat_history[].tool_callsarray<ToolCall>——NoNo——
chat_history[].tool_calls[]objectToolCall—NoYes—Bagian stabil kontrak vendor; detail bertipe baru tetap dipertahankan.
chat_history[].tool_calls[].params_as_jsonstring——NoYes——
chat_history[].tool_calls[].request_idstring——NoYes——
chat_history[].tool_calls[].tool_detailsobject or null——YesNo——
chat_history[].tool_calls[].tool_has_been_calledboolean——NoYes——
chat_history[].tool_calls[].tool_namestring——NoYes——
chat_history[].tool_calls[].typestring or null——YesNo——
chat_history[].tool_resultsarray<ToolResult>——NoNo——
chat_history[].tool_results[]objectToolResult—NoYes——
chat_history[].tool_results[].is_blockedboolean——NoNo——
chat_history[].tool_results[].is_errorboolean——NoYes——
chat_history[].tool_results[].request_idstring——NoYes——
chat_history[].tool_results[].result_valuestring——NoYes——
chat_history[].tool_results[].tool_has_been_calledboolean——NoYes——
chat_history[].tool_results[].tool_latency_secsnumber——NoNo——
chat_history[].tool_results[].tool_namestring——NoYes——
chat_history[].tool_results[].typestring——NoYes——
check_any_tool_matchesboolean——NoNo——
conversation_initiation_sourcestring——NoNo"unknown", "telegram_integration"—
dynamic_variablesobject——NoNo——
from_conversation_metadataobject or nullFromConversationMetadata—YesNo——
from_conversation_metadata.conversation_idstring——NoYes——
from_conversation_metadata.event_seqinteger——NoYes——
from_conversation_metadata.original_agent_responsestring or null——YesNo——
from_conversation_metadata.system__conversation_idstring or null——YesNo——
from_conversation_metadata.system__time_utcstring—date-timeNoYes——
namestring——NoYes——
parent_folder_idstring or null——YesNo——
tool_call_parametersobject or nullToolCallParameters—YesNo——
tool_call_parameters.parametersarray<ParameterCondition>——NoNo——
tool_call_parameters.parameters[]objectParameterCondition—NoYes——
tool_call_parameters.parameters[].evalobjectAnythingEval or ExactEval or RegexEval or LlmEval—NoYes——
tool_call_parameters.parameters[].pathstring——NoYes——
tool_call_parameters.referenced_toolobject or nullReferencedTool—YesNo——
tool_call_parameters.referenced_tool.idstring——NoYes——
tool_call_parameters.referenced_tool.typestring——NoYes"system", "webhook", "google_sheets", "client", "workflow", "api_integration_webhook", "mcp", "code"—
tool_call_parameters.verify_absenceboolean——NoNo——
tool_call_parameters.workflow_node_transitionobject or nullWorkflowTransition—YesNo——
tool_call_parameters.workflow_node_transition.agent_idstring——NoYes——
tool_call_parameters.workflow_node_transition.target_node_idstring——NoYes——
tool_call_parameters.workflow_node_transition.typestring——NoNo"node_id"—
typestring——NoYes"tool"—

Next Reply request example

Next Reply request JSON
{
  "chat_history": [
    {
      "message": "How do I return an unopened order?",
      "role": "user"
    }
  ],
  "evaluation_model": "gpt-5-mini",
  "failure_examples": [
    "I cannot help with returns."
  ],
  "name": "Helpful return reply",
  "success_condition": "Explain the return policy and give one clear next action.",
  "success_examples": [
    "You can return it within the stated window. Start here…"
  ],
  "type": "llm"
}

Simulation request example

Simulation request JSON
{
  "chat_history": [],
  "dynamic_variables": {
    "customer_tier": "standard"
  },
  "evaluation_model": "gpt-5-mini",
  "name": "Refund policy conversation",
  "simulated_user_model": "gpt-5-mini",
  "simulation_max_turns": 5,
  "simulation_scenario": "A customer asks whether an unopened order can be returned.",
  "success_conditions": [
    "The agent explains the return window and gives the next action."
  ],
  "tool_mock_config": {
    "fallback_strategy": "raise_error",
    "mocked_tool_ids": [],
    "mocking_strategy": "all"
  },
  "tool_mock_overrides": {},
  "type": "simulation"
}

Tool Invocation request example

Tool Invocation request JSON
{
  "chat_history": [
    {
      "message": "Where is order order_demo?",
      "role": "user"
    }
  ],
  "check_any_tool_matches": true,
  "name": "Look up the order",
  "tool_call_parameters": {
    "parameters": [
      {
        "eval": {
          "expected_value": "order_demo",
          "type": "exact"
        },
        "path": "order_id"
      }
    ],
    "referenced_tool": {
      "id": "tool_demo",
      "type": "webhook"
    },
    "verify_absence": false
  },
  "type": "tool"
}

Example

cURL
curl --request POST 'https://operade.ai/api/v1/workspaces/ws_demo/agent-tests' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{"type":"llm","name":"Helpful reply","chat_history":[{"role":"user","message":"How do I return an order?"}],"success_condition":"Explain the return policy clearly."}'
Request body JSON
{
  "type": "llm",
  "name": "Helpful reply",
  "chat_history": [
    {
      "role": "user",
      "message": "How do I return an order?"
    }
  ],
  "success_condition": "Explain the return policy clearly."
}

Responses

StatusDescriptionContent
4XXHandled client errorapplication/json: ErrorResponse
201Successful Responseapplication/json: object

4XX application/json response schema

Field pathTypeReferenceFormatNullableRequiredEnumDescription
$objectErrorResponse—NoYes——
errorobjectErrorItem—NoYes——
error.codestring——NoYes——
error.detailsobject——NoNo——
error.messagestring——NoYes——

201 application/json response schema

Field pathTypeReferenceFormatNullableRequiredEnumDescription
$objectSimulationTestResource or NextReplyTestResource or ToolInvocationTestResource—NoYes——
accessarray<AccessGrant>——NoYes——
access[]objectAccessGrant—NoYes——
access[].rolestring——NoYes"viewer", "editor"—
access[].user_idstring——NoYes——
access_rolestring——NoYes"admin", "creator", "viewer", "editor"—
chat_historyarray<HistoryEntry>——NoYes——
chat_history[]objectHistoryEntry—NoYes——
chat_history[].messagestring or null——YesNo——
chat_history[].rolestring——NoYes"user", "agent"—
chat_history[].tool_callsarray<ToolCall>——NoNo——
chat_history[].tool_calls[]objectToolCall—NoYes—Bagian stabil kontrak vendor; detail bertipe baru tetap dipertahankan.
chat_history[].tool_calls[].params_as_jsonstring——NoYes——
chat_history[].tool_calls[].request_idstring——NoYes——
chat_history[].tool_calls[].tool_detailsobject or null——YesNo——
chat_history[].tool_calls[].tool_has_been_calledboolean——NoYes——
chat_history[].tool_calls[].tool_namestring——NoYes——
chat_history[].tool_calls[].typestring or null——YesNo——
chat_history[].tool_resultsarray<ToolResult>——NoNo——
chat_history[].tool_results[]objectToolResult—NoYes——
chat_history[].tool_results[].is_blockedboolean——NoNo——
chat_history[].tool_results[].is_errorboolean——NoYes——
chat_history[].tool_results[].request_idstring——NoYes——
chat_history[].tool_results[].result_valuestring——NoYes——
chat_history[].tool_results[].tool_has_been_calledboolean——NoYes——
chat_history[].tool_results[].tool_latency_secsnumber——NoNo——
chat_history[].tool_results[].tool_namestring——NoYes——
chat_history[].tool_results[].typestring——NoYes——
conversation_initiation_sourcestring——NoYes"unknown", "telegram_integration"—
created_atstring—date-timeNoYes——
created_bystring——NoYes——
current_revision_idstring——NoYes——
dynamic_variablesobject——NoYes——
entity_typestring——NoYes"test"—
evaluation_modelstring——NoYes——
from_conversation_metadataobject or nullFromConversationMetadata—YesYes——
from_conversation_metadata.conversation_idstring——NoYes——
from_conversation_metadata.event_seqinteger——NoYes——
from_conversation_metadata.original_agent_responsestring or null——YesNo——
from_conversation_metadata.system__conversation_idstring or null——YesNo——
from_conversation_metadata.system__time_utcstring—date-timeNoYes——
idstring——NoYes——
namestring——NoYes——
parent_folder_idstring or null——YesYes——
revision_created_atstring—date-timeNoYes——
simulated_user_modelstring——NoYes——
simulation_max_turnsinteger——NoYes——
simulation_scenariostring——NoYes——
success_conditionsarray<string>——NoYes——
success_conditions[]string——NoYes——
tool_mock_configobjectToolMockConfig—NoYes——
tool_mock_config.fallback_strategystring——NoNo"raise_error", "call_real_tool"—
tool_mock_config.mocked_tool_idsarray<string>——NoNo——
tool_mock_config.mocked_tool_ids[]string——NoYes——
tool_mock_config.mocking_strategystring——NoNo"all", "selected", "none"—
tool_mock_overridesobject——NoYes——
typestring——NoYes"simulation"—
updated_atstring—date-timeNoYes——
failure_examplesarray<string>——NoYes——
failure_examples[]string——NoYes——
success_conditionstring——NoYes——
success_examplesarray<string>——NoYes——
success_examples[]string——NoYes——
check_any_tool_matchesboolean——NoYes——
tool_call_parametersobject or nullToolCallParameters—YesYes——
tool_call_parameters.parametersarray<ParameterCondition>——NoNo——
tool_call_parameters.parameters[]objectParameterCondition—NoYes——
tool_call_parameters.parameters[].evalobjectAnythingEval or ExactEval or RegexEval or LlmEval—NoYes——
tool_call_parameters.parameters[].pathstring——NoYes——
tool_call_parameters.referenced_toolobject or nullReferencedTool—YesNo——
tool_call_parameters.referenced_tool.idstring——NoYes——
tool_call_parameters.referenced_tool.typestring——NoYes"system", "webhook", "google_sheets", "client", "workflow", "api_integration_webhook", "mcp", "code"—
tool_call_parameters.verify_absenceboolean——NoNo——
tool_call_parameters.workflow_node_transitionobject or nullWorkflowTransition—YesNo——
tool_call_parameters.workflow_node_transition.agent_idstring——NoYes——
tool_call_parameters.workflow_node_transition.target_node_idstring——NoYes——
tool_call_parameters.workflow_node_transition.typestring——NoNo"node_id"—

Response example

Response body JSON
{
  "entity_type": "test",
  "id": "string",
  "current_revision_id": "string",
  "created_by": "string",
  "created_at": "2026-01-01T00:00:00Z",
  "updated_at": "2026-01-01T00:00:00Z",
  "revision_created_at": "2026-01-01T00:00:00Z",
  "access_role": "admin",
  "access": [],
  "name": "string",
  "parent_folder_id": "string",
  "chat_history": [],
  "dynamic_variables": {},
  "conversation_initiation_source": "unknown",
  "from_conversation_metadata": {
    "conversation_id": "string",
    "event_seq": 0,
    "system__time_utc": "2026-01-01T00:00:00Z"
  },
  "type": "simulation",
  "simulation_scenario": "string",
  "success_conditions": [],
  "simulation_max_turns": 0,
  "simulated_user_model": "string",
  "evaluation_model": "string",
  "tool_mock_config": {},
  "tool_mock_overrides": {}
}

Errors

Reviewed common and route-declared errors are summarized here; the list is not exhaustive. Start with the distributed error index to find each code's canonical domain page.

StatusCodeDescription
401unauthenticatedAuthentication is missing, invalid, or revoked.
403forbiddenThe authenticated caller does not have access to this operation.
404not_foundThe requested workspace resource was not found.
422validation_failedThe request did not pass field or business validation.
On this page
  1. Endpoint
  2. Authentication
  3. Parameters
  4. Request body
  5. application/json request schema
  6. Example
  7. Responses
  8. 4XX application/json response schema
  9. 201 application/json response schema
  10. Response example
  11. Errors
  12. Related operations