Docs

Volcengine Managed Service for Prometheus

High-availability collection solution

Scaling out agents

As your business develops, the initial number of agent shards that you deployed may not support the increasing volume of business data. The typical issues include the saturation of cluster resources for related agents and the abnormal operational status of containers. In this case, you need to scale out the agents. This topic describes how to scale out agents.

tip

When you need to scale out agents, it is recommended to first increase the number of shards for agents such as the VM Agent collector according to the description in this topic. If you need to increase the number of replicas for agents, refer to Deploying VM Agent.

Scaling out VM Agent collectors

warning

During scale-out, the rolling update of all VM Agent collectors is performed. If the update takes too long, breakpoints may occur in the monitoring data.

When you scale out the VM Agent collector and the kube-state-metrics agent, it is recommended to specify the number of shards for the kube-state-metrics agent as an integer multiple of the number of shards for the VM Agent collector to ensure load balancing. For example, maintain a 1:1 or 2:1 ratio of the number of shards for the kube-state-metrics agent to the number of shards for the VM Agent collector.

You can quickly scale out the VM Agent collector by using the deployment script. Procedure:

Download the script in your local environment. For more details, refer to Obtaining a script.

Use kubectl to connect to the Kubernetes cluster.

Run the following command to scale out the VM Agent collector:

Scaling out kube-state-metrics agents

warning

During the scale-out, the existing replicas of the kube-state-metrics agent will be sharded again before the new replicas are ready. Therefore, if the update takes too long, breakpoints may occur in the monitoring data. We recommend performing this operation during off-peak data periods.

When you scale out the VM Agent collector and the kube-state-metrics agent, it is recommended to specify the number of shards for the kube-state-metrics agent as an integer multiple of the number of shards for the VM Agent collector to ensure load balancing. For example, maintain a 1:1 or 2:1 ratio of the number of shards for the kube-state-metrics agent to the number of shards for the VM Agent collector.

You can quickly scale out the kube-state-metrics agent by using the deployment script. Procedure:

Download the script in your local environment. For more details, refer to Obtaining a script.

Use kubectl to connect to the Kubernetes cluster.

Run the following command to scale out the kube-state-metrics agent:

Use the --help parameter

You can view all the predefined fields and their meanings in the installation script by using the --help parameter.

The following section describes the meanings and examples of different fields in the installation script.

Parameters	Description
--workload	Select the agent to scale out. Available options include vmagent and kube-state-metrics.
--shards	Specify the number of shards after scale-out.
--namespace	Select the namespace of the agent.

Last updated: 2026.10.10 10:16:50