Application Performance Management Plus
Application Performance Management Plus
Document
- Documentation
Application Performance Management PlusUser guideServer monitoringUser guideService monitoring
Copy page
Download PDFUser guide
Service monitoring
Copy pageDownload PDFService monitoring
Service monitoring integrates information related to services, including Trace, Metrics, log, infrastructure monitoring, and more. It provides convenient monitoring capabilities such as service list, global service topology, service overview, and instance analysis.
Prerequisites
The service metrics and distributed tracing features require the application to integrate Trace capability. For details, see Application Integration.
Global service topology
The global topology helps you quickly organize the overall service architecture and upstream and downstream call relationships. You can also view the success rate of calls between services in real time and quickly identify problematic services.
Service topology page description:
- The service topology displays a tree diagram by default. You can switch to a star diagram as needed.
- Tree diagram: Shows hierarchical relationships, allowing you to more clearly view the call sequence and trace between services.
- Star diagram: Shows group relationships, providing a more intuitive understanding of how many service traces exist.
- Services are represented by points
- Point size: Associated with the number of times the service is called. The more calls, the larger the point.
- Point color: Associated with the service success rate. According to the threshold configured by the user in the configuration, the colors are displayed as follows:
- Green: Satisfactory
- Yellow: Average
- Red: Poor
- Call relationships are represented by lines
- Red (poor): Used to mark calls with a failure rate higher than 50%
- Click a node in the topology map to open the Service Metric Details panel on the right, which displays core metrics for the node, such as QPS, error rate, and latency, and provides the following quick navigation links:
- Service node: Service Details, Trace Analysis, Log Analysis
- Middleware node: Middleware Details
Service list
- Services integrated with monitoring are automatically added to the service list after they start. You can also manually create service information before the service starts.
- The list displays information such as service name, type, status, error rate, QPS, latency, CPU utilization, memory usage rate, and most recent view time. All metrics support sorting.
- Keyword search by service name is supported.
- Supports subscribing to, configuring, and deleting services. By configuring services, you can modify sampling configuration and interface configuration. For a detailed configuration description, see: Parameter management.
warning
- The priority of custom service configuration is higher than the Parameter configuration at the workspace level in Parameter management. For example, when both Parameter management and the service configure sampling rules, the actual sampling rules follow the custom service configuration.
- Service-level sampling configuration is only effective for the root span generated by the current service.
Service details
Description of metrics involved on the service details page:
Service overview
The service overview allows you to quickly obtain the most critical monitoring metrics for the current service, helping you confirm whether there are issues such as excessive calls, a high error rate, or high latency. Includes the following metrics:
tip
Large language model-related metrics are only displayed in the service overview of AI applications.
- Call overview: request count, error count, response time, large language model call count, total token consumption, average latency of large language model calls. Click the jump button in the upper left corner of the chart to navigate to the Trace Analysis page related to the current service. The time range is consistent with the current dashboard.
- Large language model metrics: large language model call QPS, large language model call count, token count, large language model call latency, TTFT, TPOT.
- Request metrics: request QPS, request count, status code distribution, status code request count. Click the data link in the chart to navigate to the Trace Analysis page related to the current service. The time range is consistent with the current dashboard.
- Latency metrics: average latency, latency PCT50, latency PCT90, latency PCT99. Click the data link in the chart to navigate to the Trace Analysis page related to the current service. The time range is consistent with the current dashboard.
- Error metrics: error count, error rate, errors per second. Click the data link in the chart to navigate to the Trace Analysis page related to the current service. The time range is consistent with the current dashboard.
- Log metrics: error log count, error log count per second. Click the jump button in the upper left corner of the chart to navigate to the Trace Analysis page related to the current service. The time range matches the current dashboard range.
- Container metrics: Service CPU Usage Ratio, CPU Usage, Service Memory Usage Ratio, memory usage.
- Interface ranking: Top 5 interface requests, Top 5 interface error count, Top 5 interface latency.
Topology analysis
You can view the full trace invocation status of the current service through the service topology, which supports viewing by full topology (default), upstream topology, downstream topology, or fixed hierarchy method.
- Click a node in the topology map, and the service metric details panel will pop up on the right, displaying the node's core metrics such as QPS, error rate, and latency, and providing the following quick navigation entries:
- Service node: Service details, Trace Analysis, Log analysis
- Middleware node: Middleware details
- It supports displaying corresponding colors based on the node error rate to help quickly identify the service health status. The mapping between colors and error rate thresholds is as follows:
- ColorHealth statusError rate interval (default)GreenSatisfiedLess than 10%YellowAdequate10%～50%RedRedGreater than 50%
- Custom adjustments are supported. The configuration steps are as follows:
- Click configuration > Modify settings in the upper right corner.
- Drag the threshold control to adjust the color range corresponding to different error rate intervals.
- After saving the configuration, view whether the node colors and legend are updated synchronously.
tip
The error threshold configuration on the service details page takes precedence over the workspace configuration and only applies to the current service. To adjust uniformly, go to Parameter management to modify.
- It supports configuring node color mapping rules by metric to help quickly identify exception nodes and shorten the path for service inspection and fault location. The parameters mainly include display metric, interval mode, threshold intervals, and color style.
- ParameterDescriptionDisplay metricIt supports selecting metrics such as QPS, error rate, average latency, PCT50 latency, PCT90 latency, and PCT9 latency. Topology node color is calculated based on the currently selected metric.Interval modeSupports automatic and custom modes. In automatic mode, intervals are divided by the operating system; in custom mode, users can manually enter threshold values.Threshold intervalDefine the value ranges corresponding to different colors. The intervals must be continuous and increasing; the operating system will validate them.Color settingSupports clicking on interval color blocks to modify the color, or precise configuration by entering HEX values.
- The configuration steps are as follows:
- Select the metric in the lower left corner of the page.
- Click configuration in the lower right corner of the page.
- Select the interval mode as Custom, enter the critical values for each interval, and adjust the corresponding colors as needed.
You can view the call status of all interfaces for the current service through the interface topology, which supports viewing by full topology (default), upstream topology, downstream topology, or fixed hierarchy method.
Click an interface in the topology map to view its core metrics such as QPS, error rate, and latency. Click upstream call, downstream call, or Trace Analysis to jump to the corresponding page to view specific interface details.
Instance analysis
tip
If the process startup time is not collected, - will be displayed.
- Click the host name to view the detailed host monitoring dashboard, which provides multiple monitoring metrics for CPU, memory, disk, and load, for example: CPU utilization, CPU monitoring, average load over the past 15 minutes, available memory, memory usage, disk space usage ratio, IO latency, and process memory usage distribution.
- Click view details in the Actions column of the target instance to view the basic information and monitoring metrics of the instance. Includes Overview, container metrics, process metrics, RED method, runtime monitoring, connection pool monitoring, and thread pool monitoring.
- Overview: Includes data overview, instance overview, and Container list. The data overview displays metrics such as request count, error count, and average time. Click container monitoring in the Actions column of the Container list to enter the container metrics page and view the metric data of the container to which the instance belongs.
- Container metrics: Includes data overview, Container list, and container dashboard. The container dashboard displays metrics such as the container's CPU utilization, the container's memory usage rate, the container's write IO throughput, the container's write IO QPS, and the container's FD thread count.
- Process metrics: Displays metrics for CPU, memory, file count, and context switches.
- RED method: Displays metrics such as request count, response latency, request QPS, status code distribution, error count, error rate, average latency, latency PCT90, top 5 interface requests, top 5 interface latency, and more.
- Runtime monitoring: Displays JVM monitoring metrics, Go Runtime monitoring metrics, Python Runtime monitoring-related metrics, e.g.: GC count, GC duration, memory usage rate, heap memory usage details, heap memory submit details, buffer usage details, cgo call count, HeapAlloc, and more.
- Connection pool monitoring: Displays metrics such as maximum connection count, active connection count, idle connection count, minimum idle connection count, maximum idle connection count, blocked connection count, and more.
- Thread pool monitoring: Displays metrics such as maximum thread count, core thread count, active thread count, thread pool size, thread pool task count, rejected task count, and more.
Interface monitoring
Through interface monitoring, you can view the complete list of service interfaces, and sort by QPS, overall error rate, or latency PCT90 within the selected time period. You can also view trend charts for key metrics of the selected interface.
- Call metrics: QPS, request count
- Error metrics: error rate, error count, and errors per second
- Latency metrics: mean response latency and quantile
Click an interface in the interface list to quickly drill down to the interface call details search page. You can sort and filter by latency to quickly locate slow queries, and filter by status code to quickly locate error queries.
Click TraceID to view Trace details and highlight the selected Span information.
Upstream and downstream calls
Through upstream and downstream monitoring, you can view the complete list of all upstream and downstream services and interfaces, view the average QPS, overall error rate, and latency PCT90 within the selected time period, and view trend charts for key metrics of the selected call.
- Call metrics: QPS, request count
- Error metrics: error rate, error count, and errors per second
- Latency metrics: mean response latency and quantile
Click the called interface and called service to quickly drill down to the interface call details search page.
Log analysis
Log analytics provides two methods—clustering and listing—to search for relevant logs in service logs. By using log filtering and log search features, you can query logs more efficiently.
Runtime monitoring
Java runtime metrics
For Java application services, runtime monitoring of core metrics is provided from a service perspective, including the following metrics: used heap memory, used non-heap memory, used heap memory—Top 10 processes, used non-heap memory—Top 10 processes, NewGen Size, NewGen Size—Top 10 processes, OldGen Size, OldGen Size—Top 10 processes, GC count, GC time, class loading count, thread count, and thread count—Top 10 processes.
Description of Java thread pool metrics:
Description of JVM metrics:
Golang runtime metrics
For Go application services, the following metrics are provided: goRoutine count, goRoutine count—Top 10 processes, GC count, GC count—Top 10 processes, GC total time, GC total time—Top 10 processes, single GC time—MAX/AVG/MIN, single time—PCT, cgo call count, cgo call count—Top 10 processes, HeapAlloc, HeapAlloc—Top 10 processes, HeapSys, HeapSys—Top 10 processes, HeapIdle, HeapIdle—Top 10 processes, HeapInuse, HeapInuse—Top 10 processes, HeapReleased, HeapReleased—Top 10 processes, HeapObject, and HeapObject—Top 10 processes.
Description of goruntime metrics:
Python runtime metrics
For Python application services, the following metrics are provided: CPU time, CPU utilization, memory usage, GC object count, thread count, and context switch count.
Description of Python runtime metrics:
Error analysis
Through error analysis, you can intuitively see the various types of interface errors present in the system and their distribution across different dimensions. You can also view the various exceptions in the system. The data is integrated with trace data, making it easier to view the context of errors and exceptions, analyze, and resolve interface errors and exceptions in the system.
SQL analytics
SQL analytics displays all SQL queries generated by the service. You can view SQL request trends, the SQLPattern list after removing parameters, as well as the events, failure rate, and average response time for each pattern.
Click SQLPattern to view all desensitized original SQL queries matching the pattern.
Last updated: 2026.09.16 14:37:19
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
