Docs

Volcengine Kubernetes Engine

Networking

Specifying node-level Pod networking parameters

Node-level Pod networking parameters are used to specify different Pod networking configurations for different nodes, enabling more fine-grained network isolation and resource management. This topic describes how to configure node-level Pod networking parameters in a VKE cluster and validate the result.

Background

In VKE clusters, cello within the vpc-cni add-on is used for Pod network management, such as assigning IP addresses to Pods and binding elastic network interfaces (ENIs). By default, all network configurations of cello (such as Pod subnets, security groups, ENI quotas, and network modes) are cluster‑scope configurations. All nodes in the cluster share the same network configuration (cello-config). For more details, see vpc-cni.

Cluster-level configurations meet the requirements of most scenarios. However, this unified global configuration can cause many issues when nodes in a cluster run significantly different workloads. For example, some nodes run high-density Pods and require a more aggressive IP pooling policy to accelerate Pod startup. Some nodes host specific business workloads that need to be assigned to dedicated Pod subnets or bound to exclusive security groups to meet network isolation and security compliance requirements. Nodes of different instance types have different requirements for ENI quotas.

To meet the preceding requirements, VKE supports node-level network configurations. You only need to create a custom network configuration (ConfigMap) and specify this configuration on the target node pool or node using a label to achieve different node-level configurations. This solution offers the following advantages:

Fine-grained management: Adjust network parameters at the per‑node granularity. You do not need to modify the entire cluster configuration for individual nodes. This prevents a minor change from causing a major impact.

Flexible adaptation to diverse workloads: Configure IP pooling policies, Pod subnets, and security groups per node to achieve isolation and compliance in mixed deployments.

Smooth compatibility with existing clusters: Node‑level configurations take effect in overwrite mode, without affecting existing nodes that do not have the required labels. This keeps change risks under control.

Limits

Only clusters that use the VPC-CNI network model support configuring node-level Pod networking parameters.

By default, this configuration only takes effect for newly created nodes in the node pool. To make it take effect for existing nodes, you need to restart the vpc-cni add-on Pod on the corresponding node. Restarting may affect running workloads. Therefore, we do not recommend directly enabling this feature on existing nodes hosting production workloads.

Prerequisites

A cluster that uses the VPC-CNI network model has been created. For details, see Creating managed clusters.

The vpc-cni add-on has been upgraded to v1.6.0 or later. For details, see vpc-cni.

(Optional) If you need to configure Pod security groups and subnets, they need to reside within the same VPC as the cluster. Custom security groups and subnets have been created. For details, see Creating security groups and Creating subnets.

Procedure

Connect to the target cluster using kubectl. For details, see Connecting to clusters.

Create a ConfigMap for the node‑level Pod networking parameters. The code of the sample file node-config.yaml is as follows:

apiVersion: v1
kind: ConfigMap
metadata:
name: node-config1 # The ConfigMap name.
namespace: kube-system # The namespace to which the ConfigMap belongs. You can specify a custom namespace. We recommend that you use the kube-system namespace.
data:
conf: |
{
"subnets": [ # Configure the subnets used by Pods on the node.
"subnet-2d6w87rhbgnb45********",
"subnet-zfedjv8bb8r8ob********"
],
"securityGroups": [ # Configure the security groups used by Pods on the node.
"sg-2bztk3p36bd342dx0********"
],
"poolTarget": 3, # The target number of pre-reserved Pod network resources on the node.
"poolTargetMin": 5, # The minimum number of pre-reserved Pod network resources on the node.
"poolTargetLimit": 1 # The maximum proportion of pooled resources to the available quota of the node.
}

The following table describes the key network features and parameters.

Network feature	Configuration examples	Description
Pod subnets	"subnets": [ "subnet-2d6w87rhbgnb45******", "subnet-zfedjv8bb8r8ob******" ]	Configure the subnets used by Pods on the node. We recommend keeping this configuration stable throughout the lifecycle of the node. Otherwise, Pods created at different times may use different subnets, increasing network connectivity and O&M complexity.
Pod security group	"securityGroups": [ "sg-2bztk3p36bd342dx0********" ]	Configure the security groups used by Pods on the node. We recommend keeping this configuration stable across the lifecycle of the node. Otherwise, different Pods on the node may use different security groups, resulting in inconsistent access policies. warning For this parameter, you cannot specify a managed security group. Otherwise, exceptions may occur. For more information about managed security groups, see Security group overview.
Pooling parameters	"poolTarget": 3, "poolTargetMin": 5, "poolTargetLimit": 1	cello IP pooling parameters. The system manages VPC ENIs and IPs through this parameter to accelerate Pod creation. When you configure pooling parameters, you need to balance Pod startup speed and IP resource consumption. poolTarget: The target number of pre-allocated Pod network resources on the node. cello attempts to prepare this number of available IP or ENI resources in advance, reducing the time Pods spend waiting for network resources during creation. poolTargetMin: The minimum number of pre-allocated Pod network resources on the node. When the number of available resources is lower than this value, cello attempts to supplement resources to prevent insufficient resources during high-frequency Pod creation on the node. poolTargetLimit: The percentage of the available quota of the node that can be occupied by pooled resources. Valid values: 0 to 1. A value of 1 indicates that pooled resources can occupy up to 100% of the node quota. cello calculates the resources that can be reserved based on the ENI or IP quota of the node instance type, preventing pooled resources from consuming excessive node network quota.

Run the following command to create a ConfigMap.

kubectl apply -f node-config.yaml

Add a node label to the node pool, and set its value to the ConfigMap you created in the previous step. For more information about node pool parameters, see Editing node pools.

The label key is vke.volcengine.com/vpc-cni-config.

The label value is kube-system.node-config1, where kube-system and node-config1 are the namespace and name of the ConfigMap, respectively.

warning

If the error message The specified ConfigMap is not in the correct format is returned, use a JSON validator to check the format of the conf field in the ConfigMap.

Validate results

After the configuration is complete, you can scale out the node pool to confirm whether the newly added nodes use the specified node-level networking configuration. This topic uses node-level subnets and security groups as examples.

Scale out the node pool to enable the ENIs of the newly added nodes to use the security group and subnet configured in the specified ConfigMap. For operations related to node pool scale-out, see Scaling node pools.

This topic provides an example of how to scale out a new node. As shown in the following figure, a node named node-qp6k6s is scaled out.

Check whether the ENI of the newly scaled-out node uses the specified subnet and security group.

In the left navigation bar, click Instances & Images > Instances.

Search for the target instance by node name and then click the instance name.

On the instance details page, check whether the auxiliary ENI with the volc:vke:created-by=cello label uses the specified security group and subnet on the Network Interface tab.

Schedule the business pods to the target node. Use the pod security group and subnet at the node pool level in your business scenarios.

This topic provides an example of how to configure node affinity scheduling for a Deployment. In this way, pods are scheduled to the node-qp6k6s node in the node pool with the label vke.volcengine.com/vpc-cni-config: kube-system.node-config1. For the configuration descriptions of Deployments, see Managing Deployments.

Last updated: 2026.09.03 10:38:15