Volcengine Kubernetes Engine
Volcengine Kubernetes Engine
Volcengine Kubernetes Engine
Document
Volcengine Kubernetes Engine
What's New
History feature release notes
Kubernetes version release notes
Containerd runtime release notes
Add-on release notes
Monitoring add-ons
Scheduling add-ons
DNS add-ons
Security add-ons
GPU add-ons
Sandbox add-on
Security announcements
Product Announcement
Historical announcements
Overview
Getting started
User guide
Clusters
Creating clusters
Connecting to clusters
Upgrading clusters
Node pool
Creating node pools
Scaling out/in node pools
Nodes
Authorization management
Image
Namespaces
Networking
Cluster network
Container Network
Service
LoadBalancer
CLB
Ingress
APIG Ingress
ALB Ingress
Customize ALB Ingress forwarding rules
Certificate and security
CLB Ingress
Nginx Ingress
Gateway API
Workloads
Creating workloads
Configure workloads
Pods
Configuration management
Image acceleration
Storage
FSX client
EBS Volumes
TOS Volumes
NAS volumes
FSX access methods
NFS access methods
EFS volumes
vePFS volumes
FSX access methods
vePFS Client access methods
NFS access methods
CloudFS volumes
Local volumes
Autoscale
Node autoscaler
Workload autoscaler
Horizontal Pod Autoscaler (HPA)
Cron Horizontal Pod Autoscaler (CronHPA)
Intelligent Horizontal Pod Autoscaler (IHPA)
Horizontal Pod Autoscaler based on event-driven (KEDA)
Add-on management
Network add-ons
Storage add-ons
Monitoring add-ons
Scheduling add-ons
DNS add-ons
GPU add-ons
Core add-ons
Sandbox add-on
Upgrading add-ons
Heterogeneous computing
NVIDIA GPU
mGPU scheduling
RDMA high-performance computing
Scheduling management
Topology-aware scheduling
QoS-aware scheduling
Observability
Log management
Basic observation
Setup & Configuration
Cluster observation
Storage observation
Other observation
Using metrics
Full stack observation
Alert management
Diagnosis and inspection
Node check and self-healing
Application center
Backup center
Stability
Large-scale cluster management
Best practices
Authorization
Clusters
Nodes and node pools
Networking
Storage
Scheduling
Observability
Solution
Security and compliance
API reference
Cluster management
NodePool management
Node management
Virtual nodes management
Add-on management
Tag management
Access policy management
Instance images management
Cloud native observability
Inspection and diagnosis
Task management
Volcengine Container Instance
Appendix
SDK reference
Development guide
Terraform
FAQs
General FAQs
Cluster FAQ
cluster management
cluster network
cluster upgrade
Node and Node pool FAQs
Authorization FAQs
Workload FAQs
Service and Ingress FAQs
Storage FAQs
Heterogeneous computing FAQs
Observability FAQs
Quota FAQs
Add-on FAQs
Cloud-native AI
ServingKit
LLM applications
ComfyUI application
Deploying the Al inference application through Helm
Deepseek practices
Qwen practices
Kimi practice
Seed practices
Best practices
DeepSeek model deployment
GLM model deployment
MiniMax model deployment
Qwen model deployment
Kimi model deployment
Volcengine Container Instance
What's new
Historical feature release notes
Product Announcement
Overview
Billing
User guide
Integrating with VCI
Virtual node
VKE cluster uses virtual nodes
Self-managed/third-party cluster uses virtual node
Creating an instance
General purpose instance
GPU-accelerated computing instance
VCI Agent Sandbox
Startup acceleration
E2B-based management
Upgrade guide
Networking
Monitoring
Scheduling
Container configuration
VCI Image Cache
Best practices
Security and compliance
FAQs
- Documentation
Volcengine Kubernetes EngineVolcengine Kubernetes EngineUser guideAutoscaleWorkload autoscalerHorizontal Pod Autoscaler (HPA)Achieving autoscaling by using GPU and network metrics
Copy page
Download PDFHorizontal Pod Autoscaler (HPA)
Achieving autoscaling by using GPU and network metrics
Copy pageDownload PDFAchieving autoscaling by using GPU and network metrics
In addition to the built-in CPU and memory metrics supported by Kubernetes, Volcengine Kubernetes Engine(VKE) also provides pre-configured rules for GPU and network metrics. This allows autoscaling to be triggered based on GPU usage or network traffic of containers. This topic describes how to configure an autoscaler based on the GPU or network metric.
Background information
Description
By default, a Horizontal Pod Autoscaler (HPA) supports automatic scaling only based on CPU and memory usage. It cannot meet scaling requirements in complex scenarios. For example, production environments often expect scaling based on GPU usage or network traffic of pods.
VKE provides the prometheus-adapter add-on to address this issue. It exposes native Prometheus metrics in the Custom Metrics API format and comes pre-configured with conversion rules for GPU and network metrics. After you enable cloud native observability for a cluster, you can configure automatic workload scaling for greater stability and cost-efficiency based on collected GPU and network metric data.
Autoscaling based on GPU usage and network involves the following major steps:
- Install the prometheus-adapter add-on in the cluster.
- Enable cloud native observability in the cluster and collect relevant metrics. For details, see Enabling observability and Basic observability.
- Configure an HPA to implement workload autoscaling based on GPU usage or network traffic.
Metrics
VKE supports autoscaling based on GPU and network metrics of workloads. The following table describes the preset metrics.
Metric description | Metric name | Unit |
Metric description | Metric name | Unit |
GPU usage | k8s_pod_rate_gpu_used | % |
GPU memory usage | k8s_pod_rate_gpu_memory_used | % |
GPU memory used | k8s_pod_gpu_memory_used | MiB |
Network receive throughput | k8s_pod_network_receive_throughput | % |
Network transmit throughput | k8s_pod_network_transmit_throughput | % |
tip
- In the kube-system namespace, you can check prometheus-adapter-config to verify preset metric names and the corresponding Prometheus metrics.
- In addition to the preset GPU and network metrics, you can also configure autoscaling based on custom metrics. For details, see Achieving autoscaling based on custom metrics.
Usage limits
- If multiple metrics are specified in an HPA, the calculation is done for each metric, and then the largest of the desired replica counts is chosen.
- An HPA applies only to Deployments, Statefulsets and CustomResourceDefinitions (CRDs).
- HPAs for GPU-based autoscaling support only GPU pods, not mGPU pods.
Prerequisites
- Cloud native observability has been enabled. For details, see Enabling observability.
- The target cluster has been connected by using kubectl. For more details, see Connecting to clusters.
- The nvidia-device-plugin add-on has been installed with the Install dcgm-exporter option selected for Deploy addons. For details, see nvidia-device-plugin.
- The prometheus-agent add-on has been installed with the Install node-exporter option selected for Deploy addons. For details, see prometheus-agent.
- The prometheus-adapter add-on has been installed. For details, see prometheus-adapter.
Creating through the console
- Log in to the VKE console.
- In the left navigation bar, click Cluster.
- In the cluster list, click the target cluster.
- In the left navigation bar on the cluster management page, click Autoscaling > HPA. On the page that appears, click Create HorizontalPodAutoscaler and configure parameters as needed.
- Basic information
- ParametersDescriptionParametersDescriptionNameEnter a name that complies with the displayed naming rules. The name must be unique within the same namespace.NamespaceSelect the namespace to which the autoscaler belongs. We recommend that you select a custom namespace or the default namespace created by the system. For details about namespaces, see Namespace overview.LabelAdd Kubernetes labels to specify identifying attributes of the autoscaler. Click Add new and then enter a key-value pair that complies with the on-screen requirements.AnnotationCreate Kubernetes annotations to attach non-identifying metadata to this autoscaler for efficient metadata retrieval. Annotations can contain characters that are not permitted in labels. Click Add annotation and then enter a key-value pair that complies with the on-screen requirements.
- Scaling policy
- ParametersDescriptionParametersDescriptionScaling objectSpecify the object for autoscaling. You can specify a Deployment, StatefulSet, or Custom resource object. After specifying the type of the object for autoscaling, select a specific object from the drop-down list.Monitoring metricsConfigure the metrics based on which scaling is performed. The GPU and network metrics supported in VKE include:
- GPU usage - the average GPU usage of all pod replicas for the workload. When the GPU usage reaches this specified value, scaling is triggered.
- GPU memory usage - the percentage of the total usage of GPU video memory shared by all pods to the total GPU video memory requests shared by all pods. When the GPU video memory usage reaches this specified value, scaling is triggered.
- GPU memory - the shared GPU video memory used by all pods. When the GPU video memory size reaches this specified value, scaling is triggered.
- Network receive throughput - the sum of inbound bandwidth for all containers in all pods for the workload. When the network receive throughput reaches this specified value, scaling is triggered.
- Network transmit throughput - the sum of outbound bandwidth for all containers in all pods for the workload. When the network transmit throughput reaches this specified value, scaling is triggered.
tip- An HPA configuration cannot contain duplicate scaling metrics in the same category.
- Currently, a maximum of two metrics can be included.
- For information about how to configure autoscaling based on CPU and memory usage, see Creating HPA.
Minimum number of instancesConfigure the minimum and maximum replica counts for scaling. When the policy is triggered, workload instances will scale within this range.Maximum number of instances
- Scaling rule
- ParameterDescriptionParameterDescriptionScaling policySelect Scale-out policy and Scale-in policy to configure the workload scale-in and scale-out behavior.Silence timeSpecify the cooldown period for the scale-out or scale-in policy. The value is an integer in seconds.After the system completes scaling, the scaling policy will not be executed again during the quiet time. This prevents frequent changes in the number of workload replicas that lead to system instability.Policy typeSpecify the policy type. Valid values:
- Maximum value: executes the policy that has the most impact on pod count.
- Minimum value: executes the policy that has the least impact on pod count.
- Disable: disables the scale-out policy or scale-in policy.
PolicyConfigure the scaling policy content, including the following items:- Type: specifies the type of scaling policy behavior.
- Pod: performs scaling based on the number of pods.
- Percentage: performs scaling based on the percentage of pods.
- Threshold: specifies the threshold for scaling. This parameter needs to be used together with Type. For example, if you set the Type parameter to Pod and set the threshold to 10, it indicates that 10 replicas will be scaled out or scaled in within the time interval.
- Time interval: Specify the interval at which scaling is performed. The value is an integer. Valid values: 1 to 1800. Unit: seconds.
- Click Confirm.
Use kubectl
Use the following code example to create an HPA associated with the object for autoscaling based on GPU utilization.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: gpu-hpa # The HPA name.
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment # The type of the object for autoscaling. Supported types include Deployment, StatefulSet, and CustomResourceDefinition.
name: deployment-demo # The object instance for autoscaling.
minReplicas: 1 # The minimum number of replicas.
maxReplicas: 10 # The maximum number of replicas.
metrics:
- type: Pods
pods:
metric:
name: k8s_pod_rate_gpu_used # The metric that triggers the HPA. In this example, it is pod GPU utilization.
target:
averageValue: 20 # When GPU utilization exceeds 20%, a scale-out is triggered.
type: AverageValue
Validating results
Viewing configuration
In the left navigation bar, click Autoscaling > HPA to view the configured HPA.
Validating scaling results
- When pod GPU usage increases, a scale-out will be triggered. You can view HPA information by using the following command:
- kubectl get hpa
- The expected outcome is an increase in the Deployment replicas to 2, as demonstrated below:
- NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGEgpu-hpa Deployment/example-gpu-workload 23/20 1 10 2 62s
- When the GPU usage of pods drops below the specified value, a scale-in is triggered. You can use the following command to view HPA information:
- kubectl get hpa
- The expected outcome is a decrease in the Deployment replicas to 1, as demonstrated below:
- NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGEgpu-hpa Deployment/example-gpu-workload 0/20 1 10 1 15m
References
Different versions of HPA resources provide varied levels of granular control over scaling behavior. For details, see Kubernetes documentation.
When you configure workload autoscaling to meet your business requirements, consider the Kubernetes version that you are using.
Last updated: 2026.07.24 14:51:37
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
