Docs

Volcengine Kubernetes Engine

Monitoring

Using self-built Prometheus to monitor VCI

VCI supports monitoring VCIs based on a self-built Prometheus service. This topic describes the method of configuring metric collection rules for VCI monitoring in a self-built Prometheus service.

Prerequisites

The cluster to which the VCI belongs has installed the vci-virtual-kubelet add-on. For details, see Installing add-ons.

The VCI has been created. For details, see Creating instances.

Collecting basic metrics

VCIs provide performance metrics such as CPU, memory, and network throughput. For a detailed description of the metrics, see Supported metrics.

Method 1

If Prometheus Operator is deployed in the cluster, you can collect basic metrics of VCIs by modifying the Prometheus Operator configuration.

The following example demonstrates how to collect metrics based on service discovery configured by ServiceMonitor:

Method 2

Self-built Prometheus obtains instance metrics by accessing virtual-kubelet. The method of collection configuration is the same as accessing the native Kubernetes kubelet. You can collect basic metrics of VCIs by modifying the configuration file of open-source Prometheus. The configuration file of open-source Prometheus is usually located at /etc/prometheus/prometheus.yml or in your custom configuration directory.

The following example shows how to configure collection rules for the cAdvisor metric by using the standard configuration file:

tip

For configuration instructions related to the scrape_configs parameter, see the Official Prometheus documentation.

Validating results

You can execute the command shown in the following figure on the self-built Prometheus to verify the collection results:

Collecting GPU metrics

The GPU Exporter used for VCI GPU monitoring is compatible with the monitoring metrics provided by the open-source DCGM Exporter. For a detailed description of the metrics, see Metrics supported by DCGM.

Method 1

If Prometheus Operator is deployed in the cluster, you can use the ServiceMonitor or PodMonitor of Prometheus Operator to configure service discovery for collection of VCI GPU metrics.

tip

Before collecting metrics, you need to explicitly declare the port (1025) exposed by the container in the VCI pod. The following sample code snippet provides an example of declaration:

The following code provides an example of collection via PodMonitor:

Method 2

You can configure self-built Prometheus to collect GPU metrics by modifying the configuration file of open-source Prometheus to access port 1025 of the VCI pod. The configuration file of open-source Prometheus is usually located at /etc/prometheus/prometheus.yml or in your custom configuration directory.

The following sample code shows how to configure collection rules:

Validating results

You can execute the command shown in the following figure on the self-built Prometheus to verify the collection results:

Last updated: 2026.10.08 17:31:26