Docs

Volcengine Kubernetes Engine

NLB

Configuring an NLB Service using annotations

Services support more comprehensive Layer 4 network load balancing (NLB) capabilities through annotations. This topic describes how to configure load balancing services using annotations.

warning

Annotations are case-sensitive.

Before configuring a LoadBalancer Service, be sure to carefully read Considerations for using LoadBalancer Services.

Prerequisites

To connect to an NLB instance, make sure that the cloud-controller-manager add-on is upgraded to v1.46.0 or later. For details, see Upgrading add-ons.

Creating instances

Connecting to an existing NLB instance

Create a Service and associate it with an existing NLB instance by defining the configuration in a YAML manifest. The following configuration serves as an example:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-id	String	Yes	The ID of the existing NLB instance.	No

Creating public-network NLB instances

Create a Service and provision a public load balancer instance by defining the configuration in a YAML manifest. The following configuration serves as an example:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-name	String	No	The name of the NLB instance. If you do not specify a name, it defaults to the instance ID.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-address-type	String	Yes	The network type of the NLB instance. Valid values: internet: Public network. intranet: Private network.	No
service.beta.kubernetes.io/volcengine-loadbalancer-security-groups	String	Yes	The ID of the security group associated with the NLB instance. You can specify a comma-separated list of security groups.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-zone-maps	Array of zoneMaps	Yes	The network configuration of the NLB instance. Description At least two availability zones are required.	No
service.beta.kubernetes.io/volcengine-loadbalancer-cross-zone-enabled	String	No	Whether to enable cross-AZ forwarding for the NLB instance. Valid values: true (default): Yes. false: No.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-project-name	String	No	The name of the project to which the NLB instance belongs. If this field is not specified, the project of the cluster is used.	No
service.beta.kubernetes.io/volcengine-loadbalancer-additional-resource-tags	String	No	Custom labels for the NLB instance. A label follows the key1=val1 format. You can specify multiple labels separated with commas (,).	Yes

zoneMaps

Field	Type	Required	Description	Changeable
zoneId	String	Yes	The ID of the AZ, for example, cn-beijing-a.	No
subnetId	String	Yes	The subnet ID, for example, subnet-bp12mw1f8k3jgy****.	No
eipId	String	No	EIP instance ID. For example, eip-13eoqxduidce***. If this field is not configured, a new EIP will be purchased.	No

Creating private-network NLB instances

Create a Service and provision a private load balancer instance by defining the configuration in a YAML manifest. The following configuration serves as an example:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-name	String	No	The name of the NLB instance. If no custom name is specified, it defaults to the instance ID.	No
service.beta.kubernetes.io/volcengine-loadbalancer-address-type	String	Yes	The network type of the load balancer instance. Valid values: internet: Public network. intranet: Private network.	No
service.beta.kubernetes.io/volcengine-loadbalancer-security-groups	String	Yes	The ID of the security group associated with the NLB instance. You can specify a comma-separated list of security groups.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-zone-maps	Array of zoneMaps	Yes	The network configuration of the NLB instance. Description At least two availability zones are required.	No
service.beta.kubernetes.io/volcengine-loadbalancer-cross-zone-enabled	String	No	Whether to enable cross-AZ forwarding for the NLB instance. Valid values: true: Yes. false: No.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-project-name	String	No	The name of the project to which the NLB instance belongs. If this field is not specified, the project of the cluster is used.	No
service.beta.kubernetes.io/volcengine-loadbalancer-additional-resource-tags	String	No	Custom labels for the NLB instance. A label follows the key1=val1 format. You can specify multiple labels separated with commas (,).	Yes

zoneMaps

Field	Type	Required	Description	Changeable
zoneId	String	Yes	The ID of the AZ, for example, cn-beijing-a.	No
subnetId	String	Yes	The subnet ID, for example, subnet-bp12mw1f8k3jgy****.	No
ipv4Address	String	No	The IPv4 address of the private network. For example, 192.168.1.10. If this field is not configured, the system automatically allocates an available IPv4 address from the subnet.	No

Back-end server group configuration

tip

You can configure a back-end server group when you associate an existing NLB instance or create a new NLB instance. The following example uses an existing NLB instance for demonstration.

Specifying a scheduling algorithm

An NLB service allows you to specify a scheduling algorithm for back-end server groups through annotations. The following configuration serves as an example:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-scheduler	String	Yes	The scheduling algorithm used by the back-end server group. Valid values: wrr: Weighted Round Robin (default). Back-end servers with higher weight values are polled ‌more frequently (with a higher probability)‌. wlc: Weighted Least Connections. ‌The algorithm extends the least connections algorithm‌ by assigning different weights to servers based on their ‌processing capacity‌, enabling each server to handle a number of service requests proportional to its weight. sh: Source IP Hash. ‌The algorithm uses consistent hashing‌ based on source IP addresses. Requests from the same source address ‌are directed to‌ the same back-end server.	Yes

Configuring back-end security group bypass

An NLB service allows you to bypass security groups for back-end server groups through annotations. The following configuration serves as an example:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-bypass-security-group-enabled	String	Yes	Whether to bypass back-end security groups: true: Enabled Traffic from the NLB instance is no longer subject to the security group rules of the back-end ECS or its ENI and is always allowed. false: No (default).	Yes

Configuring TCP health checking

An NLB service allows you to configure TCP health checking for back-end server groups through annotations. The following configuration serves as example:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-flag	String	No	Whether to enable health checking. Valid values: on: Yes (Default). off: Disable this setting.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-type	String	No	The health check protocol. Valid values: TCP (default) UDP HTTP warning This configuration must be consistent with the protocol configuration in the spec.ports field.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-port	String	No	The health check port. The value is an integer within the range of 1 to 65,535.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-connect-timeout	String	No	The timeout in seconds for health checks. The value is an integer within the range of 1 to 60. The default value is 3. ‌If the back-end server fails to respond within the timeout period, its health check status is marked as abnormal.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-healthy-threshold	String	No	The health threshold for health checking. The value is an integer within the range of 2 to 10. The default value is 3. The threshold is the number of consecutive health checks: if that many consecutive checks on a back-end server pass, the server is considered healthy.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-unhealthy-threshold	String	No	The unhealthy threshold for health checks. The value is an integer within the range of 2 to 10. The default value is 3. The threshold is the number of consecutive health checks: if all the consecutive checks on a back-end server fail, the server is considered unhealthy.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-interval	String	No	The time interval (seconds) of health checking. The value is an integer within the range of 1 to 300. The default interval is 10 seconds.	Yes

Configuring UDP health checking

An NLB Service allows you to configure UDP health checking for backend server groups by using annotations. The following is an example and description:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-flag	String	No	Whether to enable health checking. Valid values: on: Yes (Default). off: Disable this setting.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-type	String	No	The health check protocol. Valid values: TCP (default) UDP HTTP Note This configuration must be consistent with the protocol configuration in the spec.ports field.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-port	String	No	The health check port. The value is an integer within the range of 1 to 65,535.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-connect-timeout	String	No	The timeout in seconds for health checks. The value is an integer within the range of 1 to 60. The default value is 3. ‌If the back-end server fails to respond within the timeout period, its health check status is marked as abnormal.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-udp-request	String	No	The request string for UDP health checking. It can contain only letters and numbers. The maximum length is 64 characters. This setting and check-udp-expect must be either both empty or both non-empty.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-udp-expect	String	No	The expected response string for UDP health checking. It can contain only letters and numbers. The maximum length is 64 characters. This setting and check-udp-request must be either both empty or both non-empty.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-healthy-threshold	String	No	The health threshold for health checking. The value is an integer within the range of 2 to 10. The default value is 3. The threshold is the number of consecutive health checks: if that many consecutive checks on a back-end server pass, the server is considered healthy.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-unhealthy-threshold	String	No	The unhealthy threshold for health checks. The value is an integer within the range of 2 to 10. The default value is 3. The threshold is the number of consecutive health checks: if all the consecutive checks on a back-end server fail, the server is considered unhealthy.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-interval	String	No	The time interval (seconds) of health checking. The value is an integer within the range of 1 to 300. The default interval is 10 seconds.	Yes

Configuring HTTP health checking

An NLB Service allows you to configure HTTP health checking for backend server groups by using annotations. The following is an example and description:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-flag	String	No	Whether to enable health checking. Valid values: on: Yes (Default). off: Disable this setting.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-type	String	No	The health check protocol. Valid values: TCP (default) UDP HTTP	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-port	String	No	The health check port. The value is an integer within the range of 1 to 65,535.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-connect-timeout	String	No	The timeout in seconds for health checks. The value is an integer within the range of 1 to 60. The default value is 3. ‌If the back-end server fails to respond within the timeout period, its health check status is marked as abnormal.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-uri	String	No	The health check path. It must start with a forward slash (/). It can contain letters, digits, hyphens (-), underscores (_), forward slashes (/), periods (.), percent signs (%), question marks (?), hash signs (#), ampersands (&), and equal signs (=). The length is limited to 1 to 128 characters. If this field is not configured, the default is /.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-domain	String	No	The domain name for health checking. It must contain at least one period (.) and cannot start or end with a period (.). It can contain uppercase and lowercase letters, digits, hyphens (-), and periods (.). It cannot start or end with hyphens (-). It can contain 1 to 80 characters. If this field is not configured, health checking uses the private IP address of the back-end server.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-method	String	No	The health check method. Valid values: GET: This is the default method. To use the GET method, ensure that the server supports this method. HEAD: The HEAD method allows the server to return only the headers and thereby reduces backend overhead. To use the HEAD method, ensure that your server supports this method.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-http-code	String	No	The HTTP status code for a normal health check. Available values: http_2xx http_3xx http_4xx http_5xx	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-healthy-threshold	String	No	The health threshold for health checking. The value is an integer within the range of 2 to 10. The default value is 3. The threshold is the number of consecutive health checks: if that many consecutive checks on a back-end server pass, the server is considered healthy.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-unhealthy-threshold	String	No	The unhealthy threshold for health checks. The value is an integer within the range of 2 to 10. The default value is 3. The threshold is the number of consecutive health checks: if all the consecutive checks on a back-end server fail, the server is considered unhealthy.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-interval	String	No	The time interval (seconds) of health checking. The value is an integer within the range of 1 to 300. The default interval is 10 seconds.	Yes

Configure client IP retention

Note

To use this feature, upgrade the cloud-controller-manager add-on to v1.50.0 or later. For details, see Upgrading the add-on.

This annotation is configured at the Service level. When a single Service includes multiple ports, the configuration applies to all backend server groups of the NLB instance.

The client IP retention feature is enabled for the NLB service by default. By default, only the NLB domain name is registered, and the NLB IP is not registered. Therefore, all traffic sent from the cluster to NLB will be forwarded to the NLB instance.

After the client IP retention feature is enabled, if the client and server Pods in the VKE cluster are located on the same node, traffic may be forwarded directly within the node without passing through the NLB instance, causing network connectivity issues. When such same-node access scenarios exist, we recommend disabling the client IP retention feature. After disabling this capability, the NLB will perform SNAT to convert the client source IP to a VPC internal IP before forwarding traffic to the backend service. At the same time, the maximum number of concurrent connections for the instance will be affected.

An NLB load Service allows you to configure client IP retention for backend server groups by using annotations. The following is an example and description:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-preserve-client-ip-enabled	String	Yes	Whether to enable client IP retention. Valid values: true (default): Yes. The NLB instance forwards requests to the backend server group with the client's source IP address. The backend server can obtain the client's source IP address without additional settings. After the server group is created, you can view this information on the details page. false: Disable it. If this annotation is not configured or its value is empty, the backend server group configuration is not modified. tip When this annotation is deleted, the backend server group retains the configuration that existed before the deletion. To modify the configuration, first explicitly set this annotation to true or false, then remove the annotation. Client IP retention is unavailable in the following scenarios: If the client and the server group use different IP versions, client IP retention does not take effect. If the server group is of IP type, you cannot enable this feature.	Yes

Configure listeners

tip

A listener can be created for an existing NLB instance or a new NLB instance. The following example uses an existing NLB instance for demonstration.

Creating a TCP listener

If you do not use the service.beta.kubernetes.io/volcengine-loadbalancer-protocol-port field to configure the load balancing protocol of the listener, the system configures the listener protocol based on the spec.ports field in the YAML file. For example, if protocol is set to TCP in spec.ports, the system creates a TCP listener. The example is as follows:

Creating UDP listeners

To specify the protocol for the listener, you can use spec.ports in the YAML file instead of the annotation service.beta.kubernetes.io/volcengine-loadbalancer-protocol-port. For example, with the protocol under spec.ports set to UDP, the system will create a UDP listener accordingly. Here is an example:

Creating TLS listeners

warning

Before you create an HTTPS listener, upload a CA certificate or SSL certificate to the Certificate Center. For details, see Managing certificates, Creating SSL certificates, and Uploading SSL certificates.

When you configure a TLS listener, take note of the following information:

The port number configured using loadbalancer-protocol-port (443 in this example) must be declared in the spec.ports field. The spec.ports.protocol field for that port must be set to TCP.

If the configured port number (in this example, 443) is not declared in the spec.ports field, the TLS:443 specified in the annotation will be ignored.

If the configured port number is declared, but the spec.ports.protocol field is set to UDP, the TLS:443 specified in the annotation will be ignored.

To configure both a UDP 443 port and a TLS 443 port, set loadbalancer-protocol-port to TLS:443 and declare both the TCP 443 port and UDP 443 port in the spec field.

Annotations for LoadBalancer services can be used to create TLS listeners on NLB instances. The following configuration provides an example:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-protocol-port	String	Yes	The Layer 7 protocol and port number for the listener. In this example, it is TLS:443.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-cert-source	String	No	The server certificate source. Valid values: SSL: (default) SSL certificate from the Certificate Center. PCALeaf: Private certificate from the private CA in the Certificate Center.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-cert-id	String	No	Server certificate ID. tip CCM does not actively verify the validity of the certificate ID. If the certificate ID is invalid, the listener cannot be created successfully.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-security-policy-type	String	Yes	The security policy type of the TLS listener. Valid values: custom: Custom policy. system (default): System policy.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-security-policy-id	String	Yes	The TLS security policy. You can select a system policy or a custom policy. For a system policy, the value can be one of the following: tls_cipher_policy_1_0 tls_cipher_policy_1_1 tls_cipher_policy_1_2 tls_cipher_policy_1_2_strict tls_cipher_policy_1_2_strict_with_1_3 For a custom policy, set the value to the ID of the custom policy.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-ca-enabled	String	No	Whether to enable mutual authentication: on: It is enabled. off: (Default) Does not enable it.	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-ca-cert-source	String	No	The source of the CA certificate for mutual authentication. Valid values: PCARoot: Private CA root certificate PCASub: Private CA subordinate certificate	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-ca-cert-id	String	No	The CA certificate ID used for mutual authentication. tip CCM does not actively verify the validity of the certificate ID. If the certificate ID is invalid, the listener cannot be created successfully.	Yes

Other configurations

LB-to-pod direct access mode

warning

This feature can be enabled for an existing NLB instance or a new NLB instance. The following example uses an existing NLB instance for demonstration.

Modify this annotation with caution, as it will cause an interruption of network traffic.

If the LoadBalancer Service is associated with workloads deployed on VCI, you must enable LB-to-pod direct access mode. Otherwise, the service will be unavailable.

Annotations for LoadBalancer services can be used to configure LB-to-pod direct access mode. Here is an example:

The following table describes the annotations.

Field	Type	Required	Description	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-pass-through	String	No	Whether to enable LB-to-pod direct access mode. Valid values: true: Enabled Pods are specified as the backend servers of the load balancer. false: No. tip This configuration is optional. If this configuration is not present, the default value differs by cluster network model: For clusters that use the VPC-CNI network model, the default value is true, which means LB-to-pod direct access mode is enabled. For clusters that use the Flannel network model, the default value is false, which means LB-to-pod direct access mode is not enabled.	Yes

Last updated: 2026.09.15 10:26:27