Docs

Volcengine Kubernetes Engine

CLB

Configuring CLB LoadBalancer Services by using annotations

With annotations, Services can provide richer Layer-4 load balancing (CLB) capabilities. This topic describes how to configure load balancing services using annotations.

warning

Annotations are case-sensitive.

Before configuring a LoadBalancer Service, be sure to carefully read Considerations for using LoadBalancer Services.

Creating a service

Using an existing CLB instance

The following YAML file is an example of creating a service based on an existing LoadBalancer instance.

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-id	String	Yes	The ID of the existing LoadBalancer instance.	None	No

Using a new CLB instance

The following YAML file is an example of creating a service and a LoadBalancer instance at the same time.

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-name	String	No	Name of the LoadBalancer instance. If not specified, a name will be generated automatically.	None	No
service.beta.kubernetes.io/volcengine-loadbalancer-subnet-id	String	Yes	ID of the subnet in which the LoadBalancer instance is running.	None	No
service.beta.kubernetes.io/volcengine-loadbalancer-address-type	String	No	Network type of the LoadBalancer instance. Valid values: PUBLIC: LoadBalancer instance accessible on the Internet. PRIVATE: LoadBalancer instance accessible in private networks.	PUBLIC	No
service.beta.kubernetes.io/volcengine-loadbalancer-isp-type	String	No	The line type of the LoadBalancer instance. It includes BGP, SingleLine_BGP, Static_BGP, Fusion_BGP, ChinaMobile, ChinaTelecom, ChinaUnicom, ChinaMobile_Value, ChinaUnicom_Value, and ChinaTelecom_Value. For details, see EIP documentation. warning Except for BGP, all the line types are beta features of EIP products. To apply for these features, contact your account manager.	BGP	No
service.beta.kubernetes.io/volcengine-loadbalancer-billing-type	String	No	The billing type of the LoadBalancer instance. Valid values: 2: pay-as-you-go - specification-based billing. Select an instance specification and pay according to your actual usage time. 3: Pay-as-you-go - usage-based billing. You do not need to specify instance specifications. You pay based on the actual resources consumed by the instance. warning Pay-as-you-go - usage-based billing is a beta feature of the LoadBalancer product. To apply for this billing model, contact your account manager. There is no need for loadbalancer-spec configuration if Pay-as-you-go - usage-based billing is selected.	2	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-eip-billing-type	String	No	EIP (Elastic IP Address) billing model. Available models: 2: Pay-as-you-go - by bandwidth limit. This billing method requires a maximum bandwidth limit and incurs fees based on the usage duration, regardless of the actual traffic. 3: Pay-as-you-go - by actual traffic. This billing method requires a maximum bandwidth limit and incurs fees based on the actual outbound traffic over the Internet, regardless of the usage duration.	3	No
service.beta.kubernetes.io/volcengine-loadbalancer-bandwidth	String	No	Maximum bandwidth for Internet access of the LoadBalancer instance, measured in Mbps, available values: If eip-billing-type is set to 2, the bandwidth range is 1 to 500. If eip-billing-type is set to 3, the bandwidth range is 1 to 200.	1	No
service.beta.kubernetes.io/volcengine-loadbalancer-spec	String	No	Instance specification of the LoadBalancer. Available types: small_1: small I small_2: small II medium_1: medium I medium_2: medium II large_1: large I large_2: large II warning For details about specifications and types of LoadBalancer instances, see Product types and specifications. When loadbalancer-billing-type is set to 3, LoadBalancer instances use the Pay-as-you-go - usage-based billing type and this parameter is not supported. This annotation needs to be used together with another one named loadbalancer-sync-fields with its value set to spec.	small_1	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-sync-fields	String	Required when updating the specification of a CLB instance	Whether to update the CLB instance according to the changes of the loadbalancer-spec field when updating the YAML file. Valid values: spec: Updates the CLB instance according to the changes of the loadbalancer-spec field.	None	No
service.beta.kubernetes.io/volcengine-loadbalancer-master-zone-id	String	No	Primary availability zone ID of the LoadBalancer instance. The ID cannot be the same as that of the slave-zone-id parameter. If this parameter is missing or left empty, the availability zone where the subnet is allocated will be assigned as the master availability zone by default. tip For the availability zones supported by LoadBalancer in each region and their IDs, see Regions and Availability Zones.	None	No
service.beta.kubernetes.io/volcengine-loadbalancer-slave-zone-id	String	No	Secondary availability zone ID of the LoadBalancer instance. The ID cannot be the same as that of the master-zone-id parameter. This parameter needs to be used together with master-zone-id. Otherwise, errors will occur. If you remove this parameter or leave it empty, the following logic will be applied: For a specified region with a single availability zone, a single-zone CLB instance will be created. For a specified region with multiple availability zones, a CLB instance is created by default using randomly selected zones from the region.	None	No
service.beta.kubernetes.io/volcengine-loadbalancer-modification-protection-status	String	No	Options for modification protection: NonProtection: It disables modification protection to allow instances to be altered or removed via the console page. ConsoleProtection: It enables modification protection to prevent instances from being altered or removed via the console page. Missing this parameter or leaving it empty (default) indicates modification protection is enabled.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-eip-BandwidthPackageId	String	No	Shared bandwidth package ID. warning Effective only if service.beta.kubernetes.io/volcengine-loadbalancer-address-type=PUBLIC. After it takes effect, the billing mode (usage-based billing or subscription-based billing with tied bandwidth) previously used will be replaced.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-eni-address-number	String	No	Configure the number of private network IPv4 addresses for the LoadBalancer instance. tip It is only applicable to private CLB instances. It cannot be used simultaneously with the volcengine-loadbalancer-eni-address parameter.	None	No
service.beta.kubernetes.io/volcengine-loadbalancer-eni-address	String	No	Configure the private network IPv4 address for the LoadBalancer instance, which needs to be within the selected IPv4 subnet and not in use tip It cannot be used simultaneously with the volcengine-loadbalancer-eni-address-number parameter.	None	No

Using a new CLB instance and binding to a VCI workload

The following YAML file is an example of creating a service and binding it to a workload deployed with VCI (abbreviation for Volcengine Container Instance).

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-name	String	No	Name of the LoadBalancer instance. If not specified, a name will be generated automatically.	None	No
service.beta.kubernetes.io/volcengine-loadbalancer-subnet-id	String	Yes	ID of the subnet in which the LoadBalancer instance is running.	None	No
service.beta.kubernetes.io/volcengine-loadbalancer-pass-through	String	Yes	Whether to enable LB-to-pod direct access mode for LoadBalancer. Valid values: true: Enable this feature. Associated pods serve as the backend of the LoadBalancer. false: Disable this feature. Associated nodes serve as the backend of the LoadBalancer.	false	Yes

warning

It can only be enabled (by setting the annotation to true) for clusters using the VPC-CNI network model. Modify this annotation with caution, as it will cause an interruption of network traffic.

If the LoadBalancer Service is associated with workloads deployed on VCI, you must enable LB-to-pod direct access mode. Otherwise, the service will be unavailable.

Listener configuration

tip

A listener can be created based on an existing CLB instance or a new CLB instance. The following examples will use an existing CLB instance.

Creating a UDP listener

To specify the protocol for the listener, you can use spec.ports in the YAML file instead of the annotation service.beta.kubernetes.io/volcengine-loadbalancer-protocol-port. For example, with the protocol under spec.ports set to UDP, the system will create a UDP listener accordingly. Here is an example:

Creating a TCP listener

Creating an HTTP listener

warning

To configure an HTTP listener, upgrade the cloud-controller-manager component to v1.42.3 or later. For details, see Upgrading the component.

To configure a timeout for an HTTP listener:

Upgrade the cloud-controller-manager component to v1.50.0 or later. For details, see Upgrading the component.

If a Service includes both HTTP and TCP ports, only the timeout configurations for the HTTP listener are updated.

When you delete annotations related to listener timeout, the listener retains the configuration that was used before the annotations were deleted. For example, if the default value of an annotation is 20, and you change the value to 60 and then delete the annotation, the system keeps the value 60.

Annotations for LoadBalancer services can be used to create HTTP listeners running on CLB instances. Here is an example:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-protocol-port	String	Yes	The Layer 7 protocol and port number for the listener. You can specify multiple values separated by commas (,), for example, https:443,http:80. Warning This annotation can be altered or deleted at any time. The system will remove the old listener and rebuild a new one according to the changes when the annotation is updated. Modify this annotation with caution, as it will cause an interruption of network traffic.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-keepalive-timeout	String	No	The timeout period for persistent connections between clients and CLB. If no access requests are sent within the timeout period, CLB terminates the current connection and establishes a new connection when the next request arrives. The value is an integer. Valid values: 0 to 900. Unit: seconds. A value of 0 indicates that persistent connections are disabled.	75	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-client-header-timeout	String	No	The timeout period for reading client request headers. A client sends a request to CLB. If the client does not transmit the complete header within the timeout period, CLB returns the 408 error code, indicating a request timeout. The value is an integer. Valid values: 30 to 120. Unit: seconds.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-client-body-timeout	String	No	The timeout period for reading client request bodies. A client sends a request to CLB. If the client does not send any request body within the timeout period, CLB returns the 408 error code to indicate a request timeout. The value is an integer. Valid values: 30 to 120. Unit: seconds. This timeout applies only to the interval between two consecutive write operations, not to the entire response transmission process.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-send-timeout	String	No	The timeout period for CLB to send responses to clients. CLB sends a response to a client. If the client does not receive any content within the timeout period, the connection is closed. The value is an integer. Valid values: 1 to 3600. Unit: seconds. This timeout applies only to the interval between two consecutive write operations, not to the entire response transmission process.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-proxy-read-timeout	String	No	The timeout period for CLB to read responses from backend servers. CLB sends a request to a backend server. If the backend server that receives the request does not respond within the timeout period, the connection is closed. The value is an integer. Valid values: 30 to 3600. Unit: seconds. This timeout applies only to the interval between two consecutive write operations, not to the entire response transmission process.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-proxy-send-timeout	String	No	The timeout period for CLB to transmit requests to backend servers. CLB sends a request to a backend server. If the backend server does not receive any content within the timeout period, the connection is closed. The value is an integer. Valid values: 30 to 3600. Unit: seconds. This timeout applies only to the interval between two consecutive write operations, not to the entire response transmission process.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-proxy-connect-timeout	String	No	The timeout period for establishing connections between CLB and backend servers. If CLB does not establish a connection to a backend server within the timeout period, the connection establishment operation is terminated. The value is an integer. Valid values: 4 to 120. Unit: seconds. We recommend setting this value to be greater than the health check timeout period.	4	Yes

Creating an HTTPS listener

warning

To configure an HTTPS listener, upgrade the cloud-controller-manager component to v1.42.3 or later. For details, see Upgrading the component.

To configure a timeout for an HTTP listener:

Upgrade the cloud-controller-manager component to v1.50.0 or later. For details, see Upgrading the component.

If a Service includes both HTTPS and TCP ports, only the timeout configurations for the HTTPS listener are updated.

When you delete annotations related to listener timeout, the listener retains the configuration that was used before the annotations were deleted. For example, if the default value of an annotation is 20, and you change the value to 60 and then delete the annotation, the system keeps the value 60.

Before creating an HTTPS listener, first log in to your Volcano Engine account, click Service authorization, then click Authorize Now on the page to complete cross-service authorization.

The CA and server certificate need to be uploaded first to the CLB console or Certificate Center before getting started. For details, see Certificate management, Creating SSL certificate, and Uploading SSL certificate.

When configuring an HTTPS listener, note the following information:

The port number configured using loadbalancer-protocol-port (443 in this example) must be declared in the spec.ports field. The spec.ports.protocol field for that port must be set to TCP.

If the configured port number (in this example, 443) is not declared in the spec.ports field, the HTTP:443 specified in the annotation will be ignored.

If the configured port number is declared while the spec.ports.protocol field is set to UDP, the HTTPS:443 specified in the annotation will be ignored.

Annotations for LoadBalancer services can be used to create HTTPS listeners running on CLB instances. Here is an example:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-protocol-port	String	Yes	The Layer 7 protocol and port number for the listener. You can specify multiple values separated by commas (,), for example, https:443,http:80. Warning This annotation can be altered or deleted at any time. The system will remove the old listener and rebuild a new one according to the changes when the annotation is updated. Modify this annotation with caution, as it will cause an interruption of network traffic.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-cert-source	String	Yes	Certificate source. Available values: clb: certificate from the CLB console. cert_center: certificate from the Certificate Center.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-cert-id	String	Yes	Server certificate ID. tip CCM does not actively verify the validity of the certificate ID. If the certificate ID is invalid, the listener cannot be created successfully.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-security-policy-id	String	No	Select an encryption policy to enable TLS protocols. Valid values: default_policy: supports SSL v3, TLS v1.0, TLS v1.1, TLS v1.2. tls_cipher_policy_1_0: supports TLS v1.0, TLS v1.1, TLS v1.2. tls_cipher_policy_1_1: supports TLS v1.1, TLS v1.2. tls_cipher_policy_1_2: supports TLS v1.2. tls_cipher_policy_1_2_strict: supports TLS v1.2.	default_policy	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-ca-enabled	String	No	Whether to enable mutual authentication: on: It is enabled. off: Disable this setting.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-ca-cert-id	String	Required when mutual authentication is enabled	The CA certificate ID used for mutual authentication. tip CCM does not actively verify the validity of the certificate ID. If the certificate ID is invalid, the listener cannot be created successfully.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-keepalive-timeout	String	No	The timeout period for persistent connections between clients and CLB. If no access requests are sent within the timeout period, CLB terminates the current connection and establishes a new connection when the next request arrives. The value is an integer. Valid values: 0 to 900. Unit: seconds. A value of 0 indicates that persistent connections are disabled.	75	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-client-header-timeout	String	No	The timeout period for reading client request headers. A client sends a request to CLB. If the client does not transmit the complete header within the timeout period, CLB returns the 408 error code, indicating a request timeout. The value is an integer. Valid values: 30 to 120. Unit: seconds.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-client-body-timeout	String	No	The timeout period for reading client request bodies. A client sends a request to CLB. If the client does not send any request body within the timeout period, CLB returns the 408 error code to indicate a request timeout. The value is an integer. Valid values: 30 to 120. Unit: seconds. This timeout applies only to the interval between two consecutive write operations, not to the entire response transmission process.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-send-timeout	String	No	The timeout period for CLB to send responses to clients. CLB sends a response to a client. If the client does not receive any content within the timeout period, the connection is closed. The value is an integer. Valid values: 1 to 3600. Unit: seconds. This timeout applies only to the interval between two consecutive write operations, not to the entire response transmission process.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-proxy-read-timeout	String	No	The timeout period for CLB to read responses from backend servers. CLB sends a request to a backend server. If the backend server that receives the request does not respond within the timeout period, the connection is closed. The value is an integer. Valid values: 30 to 3600. Unit: seconds. This timeout applies only to the interval between two consecutive write operations, not to the entire response transmission process.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-proxy-send-timeout	String	No	The timeout period for CLB to transmit requests to backend servers. CLB sends a request to a backend server. If the backend server does not receive any content within the timeout period, the connection is closed. The value is an integer. Valid values: 30 to 3600. Unit: seconds. This timeout applies only to the interval between two consecutive write operations, not to the entire response transmission process.	60	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-proxy-connect-timeout	String	No	The timeout period for establishing connections between CLB and backend servers. If CLB does not establish a connection to a backend server within the timeout period, the connection establishment operation is terminated. The value is an integer. Valid values: 4 to 120. Unit: seconds. We recommend setting this value to be greater than the health check timeout period.	4	Yes

Specifying scheduling algorithm

Annotations for LoadBalancer services can be used to specify scheduling algorithms for CLB instances. Here is an example:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-scheduler	String	Yes	Configuring the scheduling algorithm for the listener. Valid values: wrr: weighted round robin. The higher the weight of the backend server, the more frequently (probabilistically) it is selected as the forwarding destination. wlc: weighted least connections. This distributes requests to the backend server with the smallest ratio of current connections to weight. sh: source IP hash. Requests with source IP addresses that have the same hash value are forwarded to the same backend server.	None	Yes

Configuring timeout

tip

Timeout is only applicable to TCP and UDP listeners.

Annotations for LoadBalancer services can be used to set timeout for CLB instances. Here is an example:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-established-timeout	String	Yes	Configure the connection timeout of the listener, currently only effective for TCP and UDP listeners. Available values: TCP: 10 to 900 seconds, default is 900 seconds. UDP: 1 to 300 seconds, default is 90 seconds.	None	Yes

Configuring maximum bandwidth

Annotations for LoadBalancer services can be used to set the maximum bandwidth for CLB instances. Here is an example:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-listener-bandwidth	String	Yes	Configure the maximum bandwidth for the listener in Mbps. -1: This listener does not exclusively occupy CLB bandwidth. It shares the non-exclusive bandwidth of the CLB instance with other listeners. Value range: 1 to the CLB bandwidth that is not used exclusively. For details about bandwidth specifications for CLB instances, see Product specifications.	None	Yes

Configuring session persistence

Annotations for LoadBalancer services can be used to configure session persistence for CLB instances. Here is an example:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-persistence-timeout	String	Yes	Configure session persistence and timeout for the listener. Sessions will be preserved based on source IPs if session persistence is configured. The range of values is 0 to 3600 seconds. Details: Setting it to 0 means session persistence is disabled. Setting it to a value from 1 to 3600 means session persistence is enabled with a specified timeout.	None	Yes

Configuring health check

Configuring TCP health checking

LoadBalancer Services support configuring health checking for TCP listeners via annotations. An example and description are as follows:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-flag	String	Yes	Whether to enable health check. Valid values: on: It is enabled. off: Disable this setting.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-interval	String	No	Health check interval. Value range: 1 to 300 Unit: seconds	2	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-connect-timeout	String	No	Health check timeout. Value range: 1–60 Unit: seconds	2	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-healthy-threshold	String	No	The number of consecutive health check requests required to determine that the back-end server is running properly. Value range: 3 to 10 Unit: times	3	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-unhealthy-threshold	String	No	The number of consecutive health check requests required to determine that the backend server is not running properly. Value range: 3 to 10 Unit: times	3	Yes

Configuring UDP health checking

LoadBalancer Services support configuring health checking for UDP listeners via annotations. An example and description are as follows:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-flag	String	Yes	Whether to enable health check. Valid values: on: It is enabled. off: Disable this setting.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-interval	String	No	Health check interval. Value range: 1 to 300 Unit: seconds	2	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-connect-timeout	String	No	Health check timeout. Value range: 1–60 unit: seconds	2	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-healthy-threshold	String	No	The number of consecutive health check requests required to determine that the backend server is running properly. Value range: 3 to 10 Unit: times	3	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-unhealthy-threshold	String	No	The number of consecutive health check requests required to determine that the back-end server is not running properly. Value range: 3 to 10 Unit: times	3	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-udp-request	String	No	Request string for health check. It is only applicable to UDP listeners. It needs to be used together with volcengine-loadbalancer-health-check-udp-expect. Only letters and numbers are allowed, with 64 characters at most.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-udp-expect	String	No	Request string for health check. It is only applicable to UDP listeners. It needs to be used together with volcengine-loadbalancer-health-check-udp-request. It can contain only letters and numbers. The maximum length is 64 characters.	None	Yes

Configuring HTTP health checking

warning

You need to upgrade the cloud-controller-manager add-on to v1.42.3 or later. For details, see Upgrading add-ons.

LoadBalancer Services support configuring health checking for HTTP listeners via annotations. An example and description are as follows:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-flag	String	Yes	Whether to enable health check. Valid values: on: It is enabled. off: Disable this setting.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-protocol-port	String	Yes	The Layer 7 protocol and port number for the listener. You can configure multiple values separated by commas (,), for example, http:80,http:8080.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-method	String	No	The method of health check. It is only applicable to HTTP/HTTPS listeners. Valid values: GET HEAD	GET	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-domain	String	No	The domain name for health check, only applicable to HTTP/HTTPS listeners. An empty domain name is allowed. If it is empty, the system will use the intranet IP of the backend server group to perform health check. If any domain name is configured, you need to ensure it is compliant with the convention, e.g., example.com.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-uri	String	No	Health check path, only applicable to HTTP/HTTPS listeners.	/	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-http-code	String	No	The status code for normal health checks. You can configure multiple HTTP status codes separated by commas (,). It is only applicable to HTTP/HTTPS listeners. Available values: http_2xx http_3xx http_4xx http_5xx	http_2xx	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-interval	String	No	Health check interval. Value range: 1 to 300 Unit: seconds	2	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-connect-timeout	String	No	Health check timeout. Value range: 1–60 unit: seconds	2	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-healthy-threshold	String	No	The number of consecutive health check requests required to determine that the backend server is running properly. Value range: 3 to 10 Unit: times	3	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-unhealthy-threshold	String	No	The number of consecutive health check requests required to determine that the backend server is not running properly. Value range: 3 to 10 Unit: times	3	Yes

Configure HTTPS health checking

Note

You need to upgrade the cloud-controller-manager add-on to v1.42.3 or later. For details, see Upgrading add-ons.

Before creating an HTTPS listener, first log in to your Volcano Engine account, click Service authorization, then click Authorize Now on the page to complete cross-service authorization.

The CA and server certificate need to be uploaded first to the CLB console or Certificate Center before getting started. For details, see Certificate management, Creating SSL certificate, and Uploading SSL certificate.

When configuring an HTTPS listener, note the following information:

The port number configured using loadbalancer-protocol-port (443 in this example) must be declared in the spec.ports field. The spec.ports.protocol field for that port must be set to TCP.

If the configured port number (in this example, 443) is not declared in the spec.ports field, the HTTP:443 specified in the annotation will be ignored.

If the configured port number is declared while the spec.ports.protocol field is set to UDP, the HTTPS:443 specified in the annotation will be ignored.

LoadBalancer Services support configuring health checking for HTTPS listeners via annotations. An example and description are as follows:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-flag	String	Yes	Whether to enable health check. Valid values: on: It is enabled. off: Disable this setting.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-protocol-port	String	Yes	The Layer 7 protocol and port number for the listener. You can specify multiple values separated by commas (,), for example, https:443,http:80.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-cert-source	String	Yes	Certificate source. Available values: clb: certificate from the CLB console. cert_center: certificate from the Certificate Center.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-cert-id	String	Yes	Server certificate ID. tip CCM does not actively verify the validity of the certificate ID. If the certificate ID is invalid, the listener cannot be created successfully.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-method	String	No	The method of health check. It is only applicable to HTTP/HTTPS listeners. Valid values: GET HEAD	GET	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-domain	String	No	The domain name for health check, only applicable to HTTP/HTTPS listeners. An empty domain name is allowed. If it is empty, the system will use the intranet IP of the backend server group to perform health check. If any domain name is configured, you need to ensure it is compliant with the convention, e.g., example.com.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-uri	String	No	Health check path, only applicable to HTTP/HTTPS listeners.	/	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-http-code	String	No	The status code for normal health checks. You can configure multiple HTTP status codes separated by commas (,). It is only applicable to HTTP/HTTPS listeners. Available values: http_2xx http_3xx http_4xx http_5xx	http_2xx	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-interval	String	No	Health check interval. Value range: 1 to 300 Unit: seconds	2	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-health-check-connect-timeout	String	No	Health check timeout. Value range: 1–60 unit: seconds	2	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-healthy-threshold	String	No	The number of consecutive health check requests required to determine that the backend server is running properly. Value range: 3 to 10 Unit: times	3	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-unhealthy-threshold	String	No	The number of consecutive health check requests required to determine that the backend server is not running properly. Value range: 3 to 10 Unit: times	3	Yes

Configuring access control

Annotations for LoadBalancer services can be used to configure access control for listeners. Here is an example:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-acl-status	String	Yes	Whether to enable access control for the listener. Valid values: on: It is enabled. off: Disable this setting.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-acl-type	String	Yes	The method to apply access control for the listener. Available methods: black: blocklist. The listener denies requests from IP addresses or subnets set in the specified access control policy group. tip If the access control policy group is empty, the listener will forward all requests. white: allowlist. The listener forwards requests from IP addresses or subnets set in the specified access control policy group. tip If the access control policy group is empty, the listener will deny all requests.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-acl-id	String	Yes	ID of the access control policy group bound to the listener. For details about policy groups, see Management of access control policy groups. You can specify at most five policy group IDs. If you specify multiple policy group IDs, separate them with commas (,).	None	Yes

Configuring Proxy Protocol to forward source IPs

Annotations for LoadBalancer services can be used to configure source IP forwarding for listeners. Here is an example:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-proxy-protocol	String	Yes	Whether to enable Proxy Protocol to forward source IPs. Available values: standard: Enable this setting. off: Disable this setting.	None	Yes

Configuring graceful connection termination

tip

You need to upgrade the cloud-controller-manager to version 1.43.0 or later. For details, see Upgrading add-ons.

LoadBalancer Services allow you to configure graceful connection termination for listeners by using annotations. If this feature is enabled, when backend servers are deleted or their health checks are abnormal, existing connections can still work normally within the graceful termination period. The following YAML content provides an example:

The following table describes the annotations.

Field	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-connection-drain	String	Yes	Whether to enable graceful connection termination. Valid values: on: It is enabled. off: Disable this setting.	None	Yes
service.beta.kubernetes.io/volcengine-loadbalancer-connection-drain-timeout	String	No	The period during graceful connection termination applies. The value is an integer within the range of 0 to 900. The time is measured in seconds. When the value is 0, a Reset packet is sent immediately to terminate the connection if the backend server is deleted or fails the health check. tip This parameter is required when graceful connection termination is enabled.	None	Yes

Other configurations

LB-to-pod direct access mode

warning

This mode can be enabled for an existing CLB instance or a new CLB instance. The following examples will use an existing CLB instance.

It can only be enabled (by setting the annotation to true) for clusters using the VPC-CNI network model. Modify this annotation with caution, as it will cause an interruption of network traffic.

If the LoadBalancer Service is associated with workloads deployed on VCI, you must enable LB-to-pod direct access mode. Otherwise, the service will be unavailable.

Annotations for LoadBalancer services can be used to configure LB-to-pod direct access mode. Here is an example:

The following table describes the annotations.

Key	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-pass-through	String	Yes	Whether to enable LB-to-pod direct access mode for LoadBalancer. Valid values: true: It is enabled. The pod is running as the direct backend of the LoadBalancer. false: Disabled.	false	Yes

Displaying the private IP of the CLB on the Internet

warning

This feature can be enabled for an existing CLB instance or a new CLB instance. The following examples will use an existing CLB instance.

This feature is only applicable to the CLB that is accessible on the Internet. For private CLB instances, only internal IPs can be viewed no matter whether the feature is enabled.

It is not applicable to IPv4/IPv6 dual-stack clusters.

Annotations for LoadBalancer services can be used to allow or disallow the display of private IPs of Internet-accessible CLB. Here is an example:

The following table describes the annotations.

Key	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-display-internal-ip	String	Yes	Whether to display the private IP address of the public CLB. Valid values: true: Enabled If the CLB can be accessed on the Internet, both the public and private IPs are displayed. false: Disabled. If the CLB can be accessed on the Internet, only the private IP is displayed. Missing this parameter indicates it is disabled. It means if the CLB can be accessed on the Internet, only the private IP is displayed.	false	Yes

The public and private IP addresses can be found on the console page or in the YAML editor after configuration is finished.

Adding extra labels to LoadBalancer services

warning

Labels can be added only when creating a new CLB instance. Doing it with an existing CLB will lead to errors.

It should be added when creating a new service. Leaving the annotation value empty indicates existing labels will be removed.

Existing extra labels cannot be updated by using annotations.

Annotations for LoadBalancer services can be used to add extra labels. Here is an example:

The following table describes the annotations.

Key	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-additional-resource-tags	String	Yes	Add extra labels for LoadBalancer. The value must be in the form of key=value. You can configure multiple labels separated by commas (,).	None	No

The extra labels of the CLB instance can be found on the CLB console page.