Volcengine Kubernetes Engine
Check item | Required | Self-healing | Self-healing action |
Check item | Required | Self-healing | Self-healing action |
GPU disconnection failure | Yes | Enable | Cordon the node |
GPU internal failure | Yes | Enable | Cordon the node |
Recoverable GPU memory failure | Yes | Enable | Cordon the node |
Unrecoverable GPU memory failure | Yes | Enable | Cordon the node |
GPU runtime failure | Yes | Enable | Cordon the node |
GPU runtime warning | Yes | Enable | Cordon the node |
GPU link failure | Yes | Enable | Cordon the node |
GPU link performance degradation | Yes | Enable | Cordon the node |
GPU driver failure | Yes | Enable | Cordon the node |
GPU driver warning | Yes | Enable | Cordon the node |
ECS instance failure | No | - | - |
kubelet failure | Yes | Enable | Restart kubelet |
Runtime failure | Yes | Enable | Restart containerd |
Time synchronization service error | Yes | Enable | Restart chrony |
Read-only node file system | No | - | - |
Node kernel failure | No | - | - |
Node kernel deadlock | No | - | - |
Check item | Required | Self-healing | Self-healing action |
Check item | Required | Self-healing | Self-healing action |
GPU disconnection failure | No | - | - |
GPU internal failure | No | - | - |
Recoverable GPU memory failure | No | - | - |
Unrecoverable GPU memory failure | No | - | - |
GPU runtime failure | No | - | - |
GPU runtime warning | No | - | - |
GPU link failure | No | - | - |
GPU link performance degradation | No | - | - |
GPU driver failure | No | - | - |
GPU driver warning | No | - | - |
ECS instance failure | No | - | - |
Node readiness | No | - | - |
kubelet failure | Yes | Enable | Restart kubelet |
Runtime failure | Yes | Enable | Restart containerd |
Time synchronization service error | Yes | Enable | Restart chrony |
Read-only node file system | No | - | - |
Node kernel failure | No | - | - |
Node kernel deadlock | No | - | - |
Parameter | Description |
Parameter | Description |
Name | The name of the node pool. A node pool name must meet the on-screen naming requirements. A node pool name must be unique within the same cluster. |
Billing method | The billing method for the ECS resources within the node pool. You can select one of the following options:
tip |
Enable management | It is enabled by default. This means that the managed node pool capability provided by VKE is enabled. Managed node pools deliver simpler and more efficient routine operations and maintenance, failure management, and resource utilization. |
Check and auto-recovery | This parameter is displayed only after the Enable managing toggle is switched on. When the managed node pool capability is enabled, the node check and self-healing feature is enabled by default. By configuring node self-healing rules, the system automatically checks for kubelet and runtime failures, GPU card status, and other node failures. When a node fails, the system promptly cordons the node or restarts the component to prevent service disruptions. For more information, see Check and self-healing overview. |
Check and auto-recovery rules | This parameter is displayed only after the Check and self-healing toggle is switched on. Configure check and self-healing rules for the node pool. The rules fall within two categories:
tip If you select Recommendation rule, the system automatically generates a check and self-healing rule in one of the following formats:
|
Autoscaler | Select this checkbox to enable node pool autoscaling. The default solution is instant autoscaling (Karpenter). You need to configure the following parameters:
warning After you enable Autoscaling, you need to configure autoscaling rules to achieve node autoscaling. For detailed instructions, see Node autoscaling. |
Availability zone | Availability Zones (AZs) are physically isolated infrastructure units within a single region, featuring independent power and networking systems. AZs communicate with each other over internal networks. Different regions have their own AZ. For details, see Regions and availability zones.
|
Node subnet | The system will display the list of available subnets based on the selected AZ. Select one or more subnets in the AZ to provide network resources for nodes.
tip
|
Instance type | The system displays a list of available instance types based on the selected subnets and defines the computing resource configuration of nodes based on the selected instance type. For detailed descriptions of instance types, see Instance specification list.
warning
|
Selected Specifications | The selected instance types for nodes in the node pool. You can select up to 10 instance types. tip
|
Pricing | This parameter is displayed after you select Spot instance for the Billing method parameter. Pricing of spot instances:
tip We recommend using this parameter together with the Spot instance compensation parameter in the Advanced settings section to perform data backup and other operations when instances are released. |
Project of ECS instances | Specify the owning project for the ECS instances corresponding to nodes in the node pool. This allows operations and maintenance, authorization, resource management, and other actions on the ECS instances at the project level. For more project operations and details, see Project management. |
High-performance computing cluster | This parameter is available if you select High Performance Computing Optimized for Instance type. Select the HPC cluster and project for the HPC-optimized nodes. If no HPC cluster is available, click Create high-performance computing cluster to create one. For details, see Creating HPC clusters. warning The availability zone of the selected HPC cluster must be within the availability zone of the current node pool subnet. |
Buy affinity group | [Beta - apply for trial] This parameter is displayed only after an instance type that supports affinity groups is selected. An affinity group is a set of ECS instances deployed physically close to each other. Instances within the group are interconnected via a high-performance network. This is suitable for computing scenarios requiring low latency and high bandwidth communication. After you select Enable, instances are purchased as an affinity group, and you need to select the specific specification from the Affinity group size drop-down list. warning
|
Number of nodes | Specify the initial number of nodes in the cluster to provide computing, storage, and other resources for running workloads.
warning The expected node count behaves differently for node operations performed in VKE compared to those in ECS. For details, see FAQ. |
More configuration | |
Container runtime | [Public beta] Select the container runtime and its version. tip VKE only supports the containerd container runtime. For more information about containerd versions, see Containerd runtime release notes. |
Node pool tags | Custom tags for classifying, retrieving, and aggregating node pools with similar characteristics from different dimensions. Node pool tags enable you to manage node pools more flexibly and efficiently. Click Add new, and enter the key and value based on prompts. |
Scheduling policy for multi-subnets | Specify scheduling priorities for multiple subnets in the node pool. Worker nodes are scheduled based on the subnet priority during scale-out. The following two policies are available:
|
Network communication mode | [Beta - apply for trial] The network communication mode between nodes. Valid values include TCP communication mode and RDMA communication mode.
tip Only instance types with vRDMA NICs can use the RDMA communication mode. |
ECS tags | VKE adds Volcano Engine platform labels to newly added ECS instances. These labels facilitate search and management of ECS instances. Click Add new, and enter the key and value based on the on-screen prompts. tip The ECS instance label will be added to newly added nodes after the node pool is scaled out. |
Deployment set | If you select the checkbox, the ECS instances that serve as nodes in the node pool will be added to the deployment set. The deployment set achieves security isolation at the physical level and distributes instances across different physical hardware, which ensures disaster recovery at the underlying layer and high availability. For details, see Overview of deployment sets. |
Parameter | Description |
Parameter | Description |
Operating system | Select an operating system image for nodes based on your business requirements. Valid values: Public images, Custom images, and Shared images.
warning
|
Install cloud assistant Agent at startup | The agent is installed by default. This setting cannot be modified. The node pool uses Agent for node management, which facilitates fault monitoring and self-healing for nodes in the node pool. |
Security reinforcement | Configure host security hardening.
|
Storage | Configure storage for the node pool. You need to configure the following parameters:
tip
tip The data reliability of local disks depends on the reliability of the physical servers. Using local disks may introduce data loss risks. If the application layer does not include a data redundancy mechanism, we recommend using data volumes for business data storage.
tip ESSD offers multiple performance levels. Different ESSD performance levels provide varying specifications for metrics including maximum IOPS, maximum throughput, and latency.
tip Extra performance for a data volume can be configured only in the following scenarios:
tip
tip Only data volume snapshots in the Available state are supported. |
Parameter | Description |
Parameter | Description |
Security group | To ensure that clusters and nodes can function, VKE provides a default security group named in the <Cluster ID>-common format. If you need to add custom security group rules, ensure the new rules do not conflict with the default ones. For more information about the default security group, see Security group settings. tip A single node pool can be associated with up to 4 custom security groups. |
Login methods | Specify the type of credentials for logging in to nodes in the clusters.
|
Parameter | Description |
Parameter | Description |
Node labels | Labels in VKE function identically to Kubernetes labels. Labels can be used to define attributes for nodes, facilitating node operations and management. A label can be added in batches to all the nodes in the node pool. Click Add node labels and enter a key and its value. warning Custom node labels cannot be the same as system labels. For details of system labels, see System labels and taints. |
Node taints | Taints allow nodes to repel specific pods and therefore prevent those pods from being scheduled to the nodes. You can add the same taint to multiple nodes in the node pool at the same time. Click Add node taints, and enter the key, value, and effect based on the prompts. You can specify one of the following effects for a taint:
warning Custom node taints cannot be the same as system taints. For a list of system taints, see System labels and taints. |
Enable auto renew for node labels and taints | Configure automatic updating for node labels and taints.
warning Disabling automatic updating for node labels and taints may cause features that use fixed node labels to behave unexpectedly. Proceed with caution. |
Kubelet configs | Specify custom kubelet parameters for nodes in the node pool to adjust node behavior. For detailed operations, see Custom kubelet parameters. |
Containerd configuration | Node pools support custom containerd configuration, which includes specifying the Container Registry proxy address for container images and disabling self-signed certificate verification. For detailed instructions, see Specifying the containerd parameters of node pools. |
Spot instance compensation | This parameter is displayed after you select Spot instance for the Billing method parameter. After the feature is enabled, new instances will be automatically created for replacement about 5 minutes before spot instance reclamation. For details, see Node pools of spot instances. |
Supplement sopt capacity by pay-as-you-go instances | This parameter is displayed after you select Spot instance for the Billing method parameter. After the feature is enabled, pay-as-you-go instances will be used for supplementation when spot instances are insufficient due to reasons such as inventory or pricing. When pay-as-you-go instances are running, even if spot instances meet the conditions, rebalancing will not be performed. Only when the node pool is being scaled in, pay-as-you-go instances are preferentially released. For details, see Node pools of preemptive instances. |
Node distribution ratio policy | This parameter is displayed after you select Spot instance for the Billing method parameter. After you enable the feature, set the proportion of pay-as-you-go instances to spot instances in the node pool. For details, see Node pools of preemptive instances. |
Custom name | Select and configure the custom name types for nodes in the node pool as needed. You can configure multiple custom name types at the same time. tip
|
Image lazy loading | [Public beta] This feature is not enabled by default. After you enable the feature, nodes in the node pool can use the container image acceleration feature on demand. For more information, see Container image lazy loading solutions. warning Only nodes that run containerd 1.4.13-vke.3 or later can use this feature. To view containerd versions, see FAQ. |
The script executed before node deployment | Enter a shell script. The script will automatically be executed before Kubernetes components are deployed on nodes. You need to implement the script commands yourself and ensure that the script supports reentrancy and retry logic. The script and the related logs can be found in the /usr/local/vke/userscripts directory on the node. This feature is typically used for preprocessing operations, such as setting environment variables and creating temporary directories. |
InitializeScript | Enter a shell script. After Kubernetes add-ons are deployed on the node, the script is automatically executed. You need to implement the script commands yourself and ensure that the script supports reentrancy and retry logic. The script and the related logs can be found in the /usr/local/vke/userscripts directory on the node. This feature is commonly used in scenarios such as modifying kubelet configurations. |
Bind EIP to node | Specify whether to automatically bind EIPs to nodes in the cluster.
warning An EIP address can be added to a shared bandwidth package only when all of the following conditions are met:
tip
|
Cordon node | Disabled by default. After you enable the cordon node feature, the nodes in the node pool are set to the cordoned and unschedulable status. Newly created pods cannot be scheduled to nodes in the node pool, but pods already running on the nodes are not affected. You can manually run the kubectl uncordon command to uncordon the nodes. Cordoning a node reduces the remaining resource quota of its cluster. If the cluster has insufficient available resources, node cordoning will affect the scheduling of new applications and the operation of existing ones. |