Docs

Volcengine Kubernetes Engine

LoadBalancer

Reusing existing LoadBalancer instances to deploy Services across clusters

Cross-cluster service deployment can provide better disaster tolerance and reliability for business, and realize high-level security isolation. However, creating an individual LoadBalancer instance to realize a load balancing service for each cluster will lead to a waste of resources. You can use a single LoadBalancer instance to provide service across multiple clusters by mounting the internal endpoints of those clusters to the instance.

Warning

The annotations in YAML files are case-sensitive.

It is highly recommended that you first read Considerations for using LoadBalancer services carefully before getting started.

Background information

Scenario 1: Forwarding traffic to a cluster and external endpoints

You can mount the services running in the cluster and the endpoints (for example, ECS) working outside the cluster onto the same CLB or NLB instance, so that the business deployed on clusters and ECS instances can share the same entry.

Scenario 2: Forwarding traffic to the services running in multiple clusters

You can mount the services running in different clusters onto an existing CLB or NLB instance to provide the same entry. The services reusing the same CLB or NLB instance need to share the same listener and back-end server group. Therefore, you need to manually configure the listener and back-end server group for the CLB or NLB instance, so that those services can reuse the group. Moreover, the listener needs to be listening to the ports on which those services are running.

Usage limits

Only the CLB or NLB instances manually created can be reused. In other words, CLB or NLB instances automatically created for LoadBalancer Services cannot be reused.

You cannot add, delete, or modify the service.beta.kubernetes.io/volcengine-loadbalancer-id field on an existing service.

You cannot add, delete, or modify the service.beta.kubernetes.io/volcengine-loadbalancer-server-group-id field on an existing service.

Prerequisites

A CLB or NLB instance, listener, and back-end server group have been created.

For information about how to configure CLB instances, see Creating CLB instances, Creating listeners, and Creating back-end server groups.

For information about how to configure NLB instances, see Creating NLB instances, Adding listeners, and Creating back-end server groups.

To connect to an NLB instance, make sure that the cloud-controller-manager add-on is upgraded to v1.45.1 or later. For details, see Upgrading add-ons.

Procedure

Reusing CLB instances

Reusing NLB instances

Use the following YAML configuration to create a service and reuse the manually created CLB instance in a VKE (Volcengine Kubernetes Engine) cluster.

The following table describes the annotations.

Key	Type	Required	Description	Default value	Changeable
service.beta.kubernetes.io/volcengine-loadbalancer-id	String	Yes	The ID of an existing CLB or NLB instance.	No	No
service.beta.kubernetes.io/volcengine-loadbalancer-server-group-id	String	Yes	Group ID and port number of the existing backend server. tip The IDs and port numbers of the backend server groups. Note that the port numbers need to be correspondingly consistent with the port under the spec.ports of the service, instead of the targetPort or nodePort. Multiple pairs of ID and port need to be separated with commas. You can add as many pairs as you want.	No	No
service.beta.kubernetes.io/volcengine-loadbalancer-weight	String	Yes	The weight on the back-end server. The value is an integer from 0 to 100. A greater weight means that the back-end server will receive more requests. A value of 0 indicates that the server will no longer accept new requests. tip For more information about weights on back-end servers, see Adding or removing back-end servers.	No	No

tip

For different network models, the objects mounted in the back-end server group of CLB or NLB instances differ:

VPC-CNI:

The LB-to-pod direct access mode is not enabled by default for connecting to CLB instances. That means the cluster will mount the node where the pod of the back-end service is located as a back-end server in the back-end server group. To enable LB-to-pod direct access, see Configuring CLB services based on annotations.

The LB-to-pod direct access mode is enabled by default for connecting to NLB instances. That means the cluster will directly mount the pod of the back-end service in the NLB server group.

Flannel:

If a CLB or NLB instance is associated with a workload that is deployed using ECS, the cluster mounts the node where the pod of the back-end service is located as a back-end server in the CLB or NLB back-end server group.

If a CLB or NLB instance is associated with a workload that is deployed using VCI, the cluster mounts the pod directly to the back-end server group of the CLB or NLB instance.

Validating results

Execute the following command to view the service created.

kubectl get svc -n default

The information below is what will be printed on the screen. The EXTERNAL-IP is the public IP address of the CLB or NLB instance to which the service is attached.

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
clb-service LoadBalancer 172.xxx.xxx.xxx 14.xxx.xxx.xxx 80:30891/TCP 44s

Execute the following command to view the service created.

kubectl describe svc clb-service -n default

The information below is what will be printed on the screen. The Annotations field displays information about the manually created CLB or NLB instance, including the instance ID and backend server group ID. This information is consistent with the CLB or NLB instance you created manually.

Name: clb-service
Namespace: default
Labels: service.beta.kubernetes.io/volcengine-loadbalancer-id=clb-22j***
Annotations: service.beta.kubernetes.io/system-volcengine-loadbalancer-last-applied-annotations:
{"service.beta.kubernetes.io/volcengine-loadbalancer-id":"clb-22j***","service.beta.kubernetes.io/volcengine-loadbalanc...
service.beta.kubernetes.io/volcengine-loadbalancer-id: clb-22j***
service.beta.kubernetes.io/volcengine-loadbalancer-server-group-id: rsp-22j***:80
service.beta.kubernetes.io/volcengine-loadbalancer-weights: 10
Selector: app=nginx
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 172.21.xxx.xxx
IPs: 172.21.xxx.xxx
LoadBalancer Ingress: 14.xxx.xxx.xxx
Port: test 80/TCP
TargetPort: 80/TCP
NodePort: test 30036/TCP
Endpoints: 192.168.0.44:80
Session Affinity: None
External Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal EnsuringLoadBalancer 2m19s (x2 over 2m22s) service-controller Ensuring load balancer
Normal EnsuredLoadBalancer 2m18s (x2 over 2m19s) service-controller Ensured load balancer

Log in to the load balancer console to view the corresponding CLB or NLB instance and back-end server group list. You can view the newly added back-end servers and their weights in the back-end server group.