VKE provides container resource view features at the kernel level to enhance the visibility of container resources. This topic provides a detailed introduction to the background and usage of enhanced container resource visibility.
Scenario introduction
By default, the container resource view feature is disabled. This means that CPU, memory, and other resource information for a container comes from the host (ECS instance) instead of the actual resource view of the container. This scenario aims to enhance container resource visibility by enabling the container resource view feature. The container resource view feature ensures that when using commands like top or free in the container to read interface data, you directly obtain container-specific resource information (not the resource information of the host).
The description of files and resource information supported by enhanced container resource visibility is as follows.
Background information
Container technology provides an environment isolation method that is different from traditional virtual machine technology. While Linux containers accelerate container packaging and startup, they also reduce the isolation level, with resource view isolation in Linux containers being a typical example of this issue.
Containers can limit resource usage (such as CPU and memory) via cgroups. However, note that when a process inside a container uses common commands like free or top to query monitoring data, it only retrieves physical server (host) metrics. It does not retrieve the actual resource usage of containers. This happens because containers do not isolate resource views for file systems like /proc and /sys.
From the perspective of applications, the runtime environment for processes in a container differs from that on a physical machine or virtual machine. Some applications encounter issues when running in containers. For example, in NGINX worker_processes auto configuration, programs that need host CPU information typically auto-detect the number of CPUs in their environment. But if a containerized process reads /proc/cpuinfo, it returns the total CPU cores of the host. This issue affects the running state of the service inside the container.
Prerequisites
VKE has been activated and a compliant cluster has been created. For specific requirements, see the "Limits" section of this topic. For more information about creating clusters, see Creating clusters. Limits
- Enhanced container resource visibility is not supported for existing nodes and existing pods in a node pool. We recommend that you create a new node pool to benefit from this feature.
- The container resource visibility enhancement feature depends on the cluster version, operating system image of nodes, and image version. The following table provides details of resource dependencies.
Procedure
Step 1: Create a node pool and enable enhanced container resource visibility
In the VKE console, create a new node pool on the node pool page. Note that in the Advanced configuration section, enter the following command in the InitializeScript field to enable the container resource visibility enhancement feature: sysctl -w kernel.cgroup_override_proc=1 >> /etc/sysctl.conf. Other parameters can be configured as needed. For details, see Creating node pools. Step 2: Create a pod and enable enhanced container resource visibility
On the Workload page of the VKE console, create a new workload (a Deployment in this example). Note that in the Advanced settings step, add the instance annotation vke.volcengine.com/host-resource-view: false. Other parameters can be configured as needed. For details, see Creating Deployments. Validating results
You can verify the result by comparing the resource information before and after enabling the enhanced container resource visibility feature. In this scenario, a Docker application has been deployed on a Linux host, and a container with a memory size of 1 GiB has been created.
- If enhanced container resource visibility is not enabled, running the free -m command in the container will return the resource information of the host (where the container resides), as shown in the figure below.
- If enhanced container resource visibility is enabled, running the free -m command in the container will return the actual resource information of the container, as shown in the figure below.
warning
Enhanced container resource visibility can meet regular requirements. However, note that this feature does not support the following special scenarios.
If the PID 1 process in the container is systemd, the monitoring data collection task may reside in a sub-cgroup of the container (instead of the root cgroup). This results in incorrectly collected data.
For example, when you log in to the container via SSH and run the cat /proc/cpuinfo command, the session process actually resides in the sub-cgroup created by systemd, and the result displayed will be incorrect.