Nginx Ingress is an implementation of Kubernetes Ingress, which enables you to manage and route Ingress traffic. This topic describes common best practices of the ingress-nginx add-on.
Background
VKE provides a community edition of the Nginx Ingress controller, which converts Ingresses into Nginx configurations and uses Nginx for Layer 7 traffic forwarding.
Common operations
Specifying the number of instances
After you install the ingress-nginx add-on in a VKE cluster, the default number of instances for the add-on is 2. You can also adjust the number of instances to achieve high availability of production.
- In the left navigation bar, click Cluster. Click the name of the target cluster to go to the cluster management page.
- In the left navigation bar of the cluster management page, click Addons. On the page that appears, click the Network tab.
- Move the pointer over the ingress-nginx add-on and click Deploy.
- In the ingress-nginx specifications dialog box, set the Number of instances parameter to the desired value, and then click Confirm.
- (Optional) When the number of ECS nodes is greater than or equal to the number of instances, you need to configure the anti-affinity scheduling policy for ingress-nginx-controller. In this way, multiple instances of the add-on are distributed across different nodes.
tip
After you install the ingress-nginx add-on, it is deployed in the cluster as a Deployment, which is named ingress-nginx-controller.
- In the left navigation bar of the cluster management page, click Workload > Deployment.
- On the Deployment page, find ingress-nginx-controller under the kube-system namespace and in the Actions column, click ... > Update.
- Modify the configurations based on the following sample figure.
- Sample figure for console operations:
Modifying resource limits
By default, for ingress-nginx-controller, the CPU upper limit is 0.5 cores, and the memory upper limit is 1 GiB. If your business traffic is high, we recommend that you set the CPU upper limit to at least 1 core and the memory limit to at least 2 GiB. You can adjust the resource limits of the ingress-nginx-controller add-on based on your business requirements.
tip
- Before you change the resource quota for ingress-nginx-controller, make sure that the number of instances is at least 2. For detailed operations, see the Specifying the number of instances section of this topic.
- After you change the configurations, perform a rolling update on the instances. In this case, business traffic may experience fluctuations, and persistent connections may be interrupted and re-established within 1 minute. Therefore, we recommend that you perform this operation during business off-peak hours.
- In the left navigation bar, click Cluster. Click the name of the target cluster to go to the cluster management page.
- In the left navigation bar of the cluster management page, click Addons. On the page that appears, click the Network tab.
- Move the pointer over the ingress-nginx add-on and click Configuration.
- In the ingress-nginx specifications dialog box, set the Nginx configuration parameter to your desired value, then click Confirm. For more parameter descriptions, see the ingress-nginx addon
Dedicated nodes
To ensure the high performance and stability of Nginx Ingress, you can deploy multiple ingress-nginx-controller instances on nodes with high specifications to avoid resource preemption between different instances and single points of failure.
The following content describes the configuration procedure to enable each ingress-nginx-controller instance to exclusively occupy one dedicated node. Two instances are used as an example.
- Create two dedicated worker nodes in the cluster. You can add two new nodes to an existing node pool, or add two dedicated nodes by creating a node pool. For details, see Adding nodes.
- Add a label with the key as ingress-nginx and value as true to each dedicated node. For detailed operations, see Label management.
- Add a taint to each dedicated node to prohibit pods without toleration from being scheduled to the dedicated nodes. Add a label with the same key and value to each taint. For detailed operations, see Taint management.
- Configure a taint toleration rule for the workload to schedule ingress-nginx-controller instances to the dedicated nodes. For detailed parameter descriptions, see the parameter descriptions of scheduling policies in Creating Deployments.
- Sample figure for console operations:
- Configure the same taint toleration rule for DaemonSets, such as log-collector. In this way, the corresponding instances can be deployed and run on the dedicated nodes.
- Configure nodeSelector to schedule ingress-nginx-controller instances only to the dedicated nodes.
- Configure an anti-affinity scheduling policy. In this way, the two instances of ingress-nginx-controller cannot be scheduled to the same node, which ensures high reliability.
- After configuration is complete, view the deployment status of ingress-nginx-controller instances.
Monitoring and alerting
You can use cloud-native observability to monitor ingress-nginx-controller and configure alerts based on your business requirements. For details, see Ingress service observability. Log persistence
By default, ingress-nginx-controller logs are not persistently stored. You need to configure and enable log persistence to persistently collect, store, and query ingress-nginx-controller logs. To better display the content of ingress-nginx-controller logs and improve readability, VKE provides ingress-nginx-controller log collection as an independent feature.
tip
- You need to activate TLS. To activate it, log in to the TLS console and follow the on-screen prompts.
- You need to install the log-collector add-on and ingress-nginx add-on. For related operations, see Installing add-ons.
- You need to create a log project and a log topic in TLS. For detailed operations, see Log projects and Log topics.
- View the logs of the ingress-nginx add-on.
- If you no longer need ingress-nginx logs, you can click the Addon log tab and then click Disable ingress access log. After you disable this feature, the system will no longer collect logs from the ingress-nginx-controller add-on.
Specifying custom native parameters
In a Kubernetes cluster, Ingresses manage externally accessible API objects in cluster services and provide Layer 7 load balancing capability. Ingresses can be configured with advanced features, such as externally accessible URLs, rewrite configuration, HTTPS service, and canary release.
For ingress-nginx-controller, VKE adopts a configuration method fully compatible with the community. For configuration descriptions, see Nginx configuration. Some advanced features and configurations natively supported by ingress-nginx can be configured and achieved through annotations. Configurations in the annotation of each Ingress YAML only apply to the current Ingress. For more information, see Annotations. Sample configuration: Specify the maximum request body size
For Nginx, when the request body size exceeds the maximum allowed value, a 413 error is returned to the client. You can specify this size by using the client_max_body_size parameter.
- To specify a global parameter for all Ingresses in the cluster, add the proxy-body-size key-value pair to the ingress-nginx-controller under kube-system in the ConfigMap.
- After you update the ConfigMap, enter the ingress-nginx-controller container under kube-system to check whether the Nginx configurations take effect.
-
- To use a custom value for only a specific Ingress, add an annotation to this Ingress.
-