Docs

Observability Platform

AI Agent observability

Hermes Agent trace monitoring

Hermes Agent is a core runtime component for LLM-based agent scenarios. It supports natural language parsing, intelligent task decomposition, LLM inference, and external tool calls. Through multi-round autonomous reasoning and closed-loop execution, Hermes Agent automatically completes various complex business tasks and provides the basic runtime capabilities for AI applications such as intelligent Q&A, automated O&M, and intelligent analytics. This topic describes how to integrate Hermes observability data into APMPlus so that each Hermes Agent execution is visible, measurable, and traceable.

Background

The APMPlus Hermes plugin is mainly used to collect conversation records, model call records, tool call records, runtime logs, and core metrics of Hermes Agent. The plugin delivers full observability of the agent execution trace, thoroughly resolving the black-box issue in agent execution processes.

The plugin requires no changes to your existing agent logic and has very low integration cost. After integration, you can observe the full lifecycle of a single request in APMPlus, covering the entire process from user input and LLM calls to external tool execution and final output. You can also continuously monitor various key runtime metrics, including token consumption, model inference duration, tool call duration, response payload size, and end-to-end overall duration.

Prerequisites

You have obtained the information about the required configuration parameters for integrating Hermes. The detailed parameters and acquisition methods are as follows:

Parameters	Required	Description
--region <region>	Yes	ID of the region for service integration, such as cn-beijing. For information about regions currently supported by Server-side Monitoring, see Server-launching region.
--appkey <appkey>	Yes	The authentication information of Server-side Monitoring. Although the App Key credential is independent of the integrated service type and data collection method, the App Key differs across workspaces. To obtain App Key, see How to obtain the AppKey for Server-side Monitoring authentication?.
--service-name <name>	Yes	The service name, such as hermes-agent. The displayed service name in Server-side Monitoring, which can be customized based on your needs.
--version <version>	No	Installs the specified plugin version. The default value is latest. Both 0.1.0 and v0.1.0 formats are supported.
--non-interactive	No	Non-interactive mode. For the first installation or a configuration change, pass --region, --appkey, and --service-name together.
--config-only	No	Updates only the Hermes configuration without updating the plugin files.
--uninstall	No	Uninstalls the plugin and clears the Hermes configuration without uninstalling Python dependencies.
--help	No	Displays script help.

Procedure

Step 1: Integrate Hermes

This topic provides three Hermes integration methods. You can select an appropriate method based on the actual scenario.

One-click integration via script
Automatic integration via Hermes
Script-based integration

Execute the following command to install the apmplus-hermes-plugin plugin with one click, and follow the operation guide to complete the configuration.

curl -fsSL "https://apmplus-hermes-plugin.tos-cn-beijing.volces.com/install.sh" | bash

After the installation is complete, manually restart Hermes to ensure that the plugin runs properly. Restart methods vary by environment, and the specific operations are as follows:

If you use Hermes Gateway, manually execute the hermes gateway restart command to make the plugin take effect.

If you use the command line, exit the current session and execute the hermes command in a new session.

Execute the following command to check the plugin status. If the plugin list contains apmplus_hermes_plugin and its status is enabled, the integration succeeds.

hermes plugins list

Step 2: View observability data

Interact with Hermes via WebUI or other methods. After the conversation, navigate to the APMPlus console to view the observability data.

Navigate to Service List and search for the integrated Hermes by service name.

Click the service name to access the service details page. On the Trace analysis tab, view the Span list corresponding to the Hermes service.

Click the target trace ID and switch to the LLM data or Properties tab to view the details of this request, such as trace details, request model, token consumption, and LLM, tool, and agent turn duration and token consumption.

Switch to the Detailed logs of traces tab to view detailed logs of the Hermes runtime process.

What to do next

Navigate to Preset dashboard, select Hermes observational disk, and view the preset dashboard provided by APMPlus for Hermes.

tip

If the preset dashboards do not meet your monitoring needs, you can also create custom dashboards based on your actual requirements.

Navigate to the Alerting task page and click Create alerting task. You can configure automated alert monitoring for Hermes observability metrics based on your needs, such as token consumption, session lag, LLM duration, tool duration, agent turn duration, and error calls. Configuring alert tasks helps monitor target service metrics for exceptions and ensures timely notifications, helping you keep track of Hermes operational status.

tip

Alert tasks allow you to monitor corresponding metric values. An alert notification is automatically sent when the configured conditions are met. For detailed information about alert tasks, see Alert tasks.

The following management operations are available for the apmplus-hermes-plugin plugin.

Update plugin
Modify configuration
Uninstall plugin
Common toggles
Execute the plugin installation command again. After completion, restart Hermes to update the plugin.
If the plugin has been installed, only the plugin itself is updated by default, and the existing region, appkey, service name, and other configurations will be retained. Execute the following command:

curl -fsSL "https://apmplus-hermes-plugin.tos-cn-beijing.volces.com/install.sh" | bash

To update the plugin to a specified version, for example, version 0.1.0, execute the following command:

curl -fsSL "https://apmplus-hermes-plugin.tos-cn-beijing.volces.com/install.sh" \
| bash -s -- --version 0.1.0
To modify the reporting configuration without updating the plugin file, execute the following command:

curl -fsSL "https://apmplus-hermes-plugin.tos-cn-beijing.volces.com/install.sh" \
| bash -s -- --config-only --region cn-beijing --appkey xxxxxxxxx --service-name hermes-agent

To update the plugin and overwrite the configuration, execute the following command:

curl -fsSL "https://apmplus-hermes-plugin.tos-cn-beijing.volces.com/install.sh" \
| bash -s -- --region cn-beijing --appkey xxxxxxxxx --service-name hermes-agent
To uninstall the apmplus-hermes-plugin plugin, execute the following command and restart Hermes in sequence.
warning
Uninstalling the plugin deletes the plugin directory and removes the plugin name from plugins.enabled, plugins.disabled, and the apmplus_hermes_plugin configuration in Hermes.
Since Python dependencies may be reused by other plugins, they will not be uninstalled.

curl -fsSL "https://apmplus-hermes-plugin.tos-cn-beijing.volces.com/install.sh" \
| bash -s -- --uninstall
You can disable input and output body collection. Sensitive fields are masked by default. After disabling, common fields such as Token, Password, Secret, Authorization, api_key, and other fields will not be directly exposed in plain text.

apmplus_hermes_plugin:
trace_content: false

You can disable log or metric collection. Log or metric collection is enabled by default. You can disable this configuration based on your needs.

apmplus_hermes_plugin:
logs: false
metrics: false

Last updated: 2026.07.09 20:16:23