Docs

Code Pipeline

AI application

Scaling

This article introduces how to scale the number of replicas for AI applications.

Background information

The AI application provides two methods: manual scaling and automatic scaling:

When scaling an application is an urgent requirement, for example, during a sudden traffic surge, you can select the manual scaling method.

When application scaling is a non-urgent requirement, such as periodic traffic peaks, you can select the automatic scaling method.

Prerequisites

The AI application has been created and deployed, and the application must be deployed on a Volcengine Kubernetes Engine (VKE) cluster. For specific operations, refer to create and deploy AI application.

Procedure

Select AI application from the left navigation bar.

On the AI application page, select the target AI application, click the application card, and enter the basic information tab for the current application.

Manual scaling

warning

If you configure both an automatic scaling policy and manual scaling, the results of manual scaling may be overridden by automatic scaling, so manual scaling may not take effect.

On the Basic Information tab of the current AI application, click Manual Scaling in the upper right corner.

In the pop-up dialog box, set the target number of instances.

Click confirm to start scaling the application.

Auto scaling

tip

Each application only supports adding one scaling policy.

Switch to the Elastic Scaling tab of the current application, and click Configure Elastic Scaling.

In the pop-up Elastic scaling configuration panel, configure the relevant information as required.

Configuration Item	Descriptions
Number of instances	Settings for the minimum and maximum number of instances allowed. When the policy is triggered, instances will scale within this range.
Scaling metrics	Configure monitoring metrics for scaling. The system determines whether scaling is required based on these metrics. Please configure at least one scaling metric. Scaling metrics support: CPU utilization: Calculated as the percentage obtained by dividing the total CPU usage of all Pod instances by the total Pod instance CPU requests, this metric is used for monitoring. When CPU utilization reaches the target value, scaling is triggered. Memory utilization: Calculated as the total memory usage of all Pod instances / the total memory requests of all Pod instances as a percentage, used as a monitoring metric. When memory utilization reaches the target value, scaling is performed. GPU Utilization: The DCGM_FI_PROF_SM_ACTIVE metric reflects GPU utilization, representing the proportion of time during a given interval when at least one thread block is active on a Streaming Multiprocessor (SM). When GPU utilization reaches the target value, scaling is performed.
Advanced settings	An optional configuration Cooldown window time: When scaling operations occur, the automatic scaling algorithm selects the highest expected number of instances within the past [input value] time period. This configuration is designed to prevent excessively frequent fluctuations in the number of replicas. For example: The cooldown window time is 300 s. When scaling is triggered, the system first observes for 300 seconds, then selects the highest expected instance Value and performs scaling accordingly. Expansion rate: The maximum number of instances that can be expanded within a specified time period. Scale-in rate: The maximum number of instances that can be scaled in within a specified time period. Disable scaling in: Once enabled, instances of the application will never be scaled in, effectively preventing business risks caused by scaling in during peak traffic periods. Disabled by default.

After configuration is complete, click confirm.

After configuring and enabling the auto scaling policy, you can verify whether the auto scaling policy works as expected by observing the number of instances in environment > instance list.

Related Operations

After the application's auto scaling policy configuration is complete, you can view the details of the configured policy on the Elastic Scaling page. Enable, disable, edit, or delete automatic scaling policies as needed

Last updated: 2025.09.10 16:17:53