Login to follow
Agent Monitor

Agent Monitor (ODC)

Stable version 0.1.3 (Compatible with ODC)
Uploaded on 22 May (3 weeks ago) by Truewind
Agent Monitor

Agent Monitor (ODC)

Documentation
0.1.3

Agent Monitor uses three probe pairs. Each pair has a Start action and an End action. Place the Start probe at the beginning of the step you want to observe, and the End probe immediately after it completes.

The three probe pairs are:

  • StartAgentCall / EndAgentCall: wraps the full agent execution
  • StartLLMCall / EndLLMCall: wraps each individual LLM call
  • StartAgentToolCall / EndAgentToolCall: wraps each tool call invoked by the agent

Instrumenting the Agent Flow

The agent flow is the main Server Action that runs your agent. It typically contains: StartAgentCall, BuildMessages, one or more LLM calls, tool dispatching, and a final response step.

Step 1 — Add StartAgentCall at the beginning of the flow, before any processing.

Parameters:

  • AgentName: the name of your agent (e.g., "FAQ Agent")
  • Parameters: list of input parameters as name/value pairs
  • DateTime: CurrDateTime()
  • SessionId: the current session identifier (for session-based agents)

StartAgentCall returns an AgentCallId. Store this value — it is required by all subsequent probes in this flow.

Step 2 — Add StartLLMCall before each CallModel (LLM invocation).

Parameters:

  • AgentCallId: StartAgentCall.AgentCallId
  • ModelName: the model identifier (e.g., "gpt-4.1-mini")
  • InputMessages: the message list passed to the model (e.g., BuildMessages.Messages)
  • StartedOn: CurrDateTime()

StartLLMCall returns an LLMCallId. Store this value — it is required by EndLLMCall and by any tool calls triggered by this LLM response.

Step 3 — Add EndLLMCall immediately after CallModel returns.

Parameters:

  • LLMCallId: StartLLMCall.LLMCallId
  • ResponseOn: CurrDateTime()
  • Response: the model response content (e.g., CallModel.Messages.Current.Content.Current...)
  • InputTokens: CallModel.ModelUsage.InputTokens
  • ResponseTokens: CallModel.ModelUsage.ResponseTokens

Step 4 — Add EndAgentCall at the end of the flow, after the final response is ready.

Parameters:

  • AgentCallId: StartAgentCall.AgentCallId
  • AgentResponse: the final output of the agent
  • DateTime: CurrDateTime()
  • IsError: set to True in error-handling branches to flag failed executions

Instrumenting Tool Calls

Each tool is a separate Server Action called by the agent after an LLM response requests it. To link tool calls to the LLM call that triggered them, add LLMCallId as an input parameter to each tool Server Action.

Step 1 — Add LLMCallId as an input parameter to the tool Server Action. Pass this value from the agent flow when dispatching the tool call.

Step 2 — Add StartAgentToolCall at the beginning of the tool Server Action.

Parameters:

  • LLMCallId: LLMCallId (the input parameter added in step 1)
  • ToolName: the name of the tool (e.g., "SearchFAQ")
  • ToolTypeId: Entities.ToolType.Function
  • Parameters: list of tool input parameters as name/value pairs

StartAgentToolCall returns an AgentToolCallId. Store this value for EndAgentToolCall.

Step 3 — Execute the tool logic normally.

Step 4 — Serialize the tool output to JSON before logging (e.g., using JSONSerialize).

Step 5 — Add EndAgentToolCall after the tool logic completes.

Parameters:

  • ToolCallId: StartAgentToolCall.AgentToolCallId
  • ToolResponse: the serialized JSON output of the tool
  • DateTime: CurrDateTime()