The AI application monitoring probe supports observability for large language model (LLM) applications. After the self-developed Python probe is integrated with LLM applications, you can view a list of sessions, session details, and related LLM trace details that meet specific query conditions within a selected time range. This topic describes how to view LLM session analysis and manage session data.

Prerequisites

Your LLM application has been integrated with Server-side Monitoring, and auto-instrumentation is enabled. For specific steps, see Python application integration.

View session details

Step 1: Integrate LLM session analysis

Ensure that your LLM application is successfully integrated with Server-side Monitoring with auto-instrumentation enabled, and that LLM traces and metrics are automatically generated. Then, pass session analysis metadata (session_id and user_id) from your application code. For example, in Langchain, manually add session_id and user_id to config.metadata when you call a chain/agent. Sample code:

Step 2: View LLM session data

Log in to the AI application monitoring console, and select the project and region in the top menu bar.

In the left navigation bar, select Session Analysis to go to the LLM session list.

To obtain the list of target sessions, filter sessions by workspace, time range, tags, session ID, total tokens, user ID, and other conditions. From the session list, you can view basic session information, such as session ID, duration, total tokens, input tokens, output tokens, trace count, user ID, and start time.

Click the name of the target session or the Details button in the Actions column on the right of the target session. You can then view the data of the specified session on the Session Data tab, such as the total tokens, number of traces, start time for each trace, input, output, trace ID, user ID, token count, and model name.

Click the trace ID to go to the LLM trace analysis page to view detailed information about LLM trace analysis. For trace analysis details, see Trace analysis.

Step 3: View insight details

On the Insight details tab, you can view the session insight details, including insight time, insight result, evaluation reason, and associated insight tasks. Click the task name to navigate to the task details page and view task information. For details, see Insight tasks.

Export session data

Session analysis supports exporting both the session list data and the detailed trace data of a specific session.

Export session list data

Export session details data

Go to the session management page. Select the sessions you wish to export, click the Export button in the upper-right corner of the page, and the basic information data related to the sessions will be exported to a CSV file based on the session list.

tip

You can export at most 100 records at a time.

Diagnose a session

Trace and Session data of the current session can be temporarily used for diagnosis and analysis, and detailed diagnosis reports are provided.

Go to the Session analysis list, click Session diagnostics in the Actions column of the target session, and confirm the prompt. You are redirected to the Observational diagnostics page, and a session diagnosis task starts automatically.

When the task status is "Completed", the diagnosis is complete, and the report is generated. Click View report in the Actions column on the right of the target diagnosis task to view the diagnosis report details of the session. For more details, see Observability diagnosis.

Observability Platform