VKE clusters expose metrics of control plane add-ons. This helps you use self-managed Prometheus to collect metrics and build a cluster control plane monitoring dashboard. As a result, the metrics are integrated with your self-managed monitoring system. This topic describes how to enable metric collection for cluster control plane add-ons.

Usage limits

The Kubernetes version of the cluster must meet the following requirements: If the cluster version is outdated, upgrade it first. For details, see Upgrading clusters.

Kubernetes v1.24: v1.24.17-vke.44 and later

Kubernetes v1.26: v1.26.15-vke.34 and later

Kubernetes v1.28: v1.28.15-vke.27 and later

Kubernetes v1.30: v1.30.4-vke.12 and later

Kubernetes v1.32: v1.32.3-vke.7 and later

Other later Kubernetes versions released by VKE

You can collect metrics only for the following managed add-ons: kube-apiserver, etcd, kube-scheduler, kube-controller-manager, and cloud-controller-manager in the cluster control plane. You cannot collect metrics for other managed add-ons. For related add-on metrics, see VKE metrics.

Prerequisites

Self-managed Prometheus can be deployed either inside or outside the cluster to be monitored.

When a self-managed Prometheus is deployed outside the cluster to be monitored, make sure that it can access the API server of the VKE cluster and has read permissions on /metrics.

Procedure

Step 1: Enable metric collection for control plane add-ons

Log in to the VKE console.

In the top navigation bar, select the region where your business is located.

In the left navigation pane, click Cluster. Click the name of the target cluster to enter its management page.

In the left navigation pane, click Operation and maintenance configuration. In the Custom metric collection configuration section, enable Metric collection from control plane components.

Step 2: Configure collection rules

Before you use self-managed Prometheus to collect metrics from cluster control plane add-ons, you need to configure metric collection jobs for control plane add-ons in the Prometheus configuration file prometheus.yaml. The sample code is as follows:

Cluster internal monitoring

If your self-managed Prometheus is deployed inside the cluster to be monitored, you can collect data of control plane add-ons based on the following sample collection rule code.

APIServer

ETCD

Kube-Scheduler

Kube-Controller-Manager

Cloud-Controller-Manager

The collection rule sample code is as follows:

Cluster external monitoring

If your self-managed Prometheus is deployed outside the cluster to be monitored, it cannot automatically mount ServiceAccount credentials or access internal Kubernetes services as it would inside the cluster. The configuration method is as follows:

Manually specify the API server address of the VKE cluster to connect external Prometheus to the cluster.

Manually provide authentication credentials, including a CA certificate and a bearer token, to pass the permission check of the Kubernetes API.

Perform node-based service discovery to automatically obtain the addresses of all nodes in the cluster, and subsequently collect node-related metrics.

The configuration sample code is as follows:

Validate the result

Log in to the self-managed Prometheus console and switch to the Graph page.

Enter the up command to view whether the specified collection task functions as expected.

The expected result is as follows. Take the API server observability task as an example. up{instance="xxx.xxx.xxx.xxx:6443", job="api-server-demo"} indicates the corresponding collection task. When its value is 1, it indicates the task is running as expected.

Enter the following metrics to check whether the data can be displayed as expected.

apiserver_request_total{job="api-server-demo"}

The expected result is similar to the following one. If metrics and data can be queried as expected, the self-managed Prometheus can collect metrics of the core components from the cluster.

Volcengine Kubernetes Engine