Docs

Elastic Compute Service

Kernel fine-tuning

Modifying timeout parameters for NVMe disks in Linux operating system

This topic describes how to configure the timeout periods for NVMe disks on a Linux instance mounted with NVMe disks.

warning

This practice is only applicable to instances running the Linux operating system and using NVMe disks, such as i3s, ebmg3id, and ebmhpcpni2l instances. For details of instance types, refer to Instance types.

Parameter description

For instances that use Nonvolatile Memory Express (NVMe) disks, you can configure the following parameters to set the timeout periods for NVMe disks to process I/O commands or Admin commands.

nvme_core.io_timeout

This parameter specifies the timeout period for I/O commands of a device (commands for data transmission operations, such as reading from or writing to a disk). The unit is seconds, with a default value of 30 seconds.

When the driver detects a timeout of an I/O operation, it will wake up the timeout handler (a mechanism or module for handling timeout errors). Based on the error handling policy, the handler will decide to retry the I/O operation, return failure information, or perform device recovery.

nvme_core.admin_timeout

This parameter specifies the timeout period for Admin commands of a device (commands for a host to manage and control SSD operations, such as creating I/O submission queues). The unit is seconds, with a default value of 60 seconds.

When the driver detects a timeout of an Admin operation, it will wake up the timeout handler (a mechanism or module for handling timeout errors). Based on the error handling policy, the handler will decide to retry the I/O operation, return failure information, or perform device recovery.

Applicable scenarios

Accelerate I/O response

For applications requiring fast responses, you can appropriately reduce the timeout period to improve the response speed of I/O operations.

Improve resource utilization

If resources are tight and you need to improve resource utilization, you can appropriately reduce the timeout period. This allows faster release of resources occupied by timed-out I/O operations. In turn, other tasks or I/O operations can obtain resources promptly, thereby improving the overall efficiency of the system.

Increase I/O throughput

To increase the I/O throughput of NVMe disks, you can appropriately extend the timeout period. This allows more I/O operations to be performed simultaneously, thereby making full use of the parallel processing capability of NVMe devices and improving data transmission speed and overall performance.

Increase reliability and data integrity

For scenarios requiring high reliability and data integrity, you can appropriately extend the timeout period to ensure that I/O operations have sufficient time to complete, thereby reducing data loss or errors caused by timeouts.

warning

The principle of the timeout parameter is to increase delay tolerance, which means delaying the sending of abort commands. If you specify an excessively long timeout period and the disk remains unresponsive afterward, the I/O operation of the NVMe disk will not be aborted for a long time, nor will retries be performed. This will cause the application to get stuck in this I/O operation and run abnormally.

Procedure

Step 1: Configure the timeout parameters

Execute the following command to check whether the timeout parameters are configured in the grub file.

grep -r "timeout" /etc/default/grub

If no value is displayed in the output, execute the following command to add timeout parameters to the grub file.

sed -i "/GRUB_CMDLINE_LINUX=/s/\"$/ nvme_core.io_timeout=4294967295 nvme_core.admin_timeout=4294967295 \"/" /etc/default/grub

tip

Replace 4294967295 in the command with the actual timeout periods based on your needs. The unit is seconds.

If the following output is displayed, execute the following command to modify the timeout parameter values.

Execute the following command to open the grub file.

vim /etc/default/grub

Press the i key to enter edit mode and modify the values of the nvme_core.io_timeout and nvme_core.admin_timeout parameters.

tip

Modify the timeout periods based on your needs. The unit is seconds.

Press Esc, enter :wq, and then press Enter to save and close the file.

Depending on the operating system and boot mode type, execute the following commands to apply the timeout parameter configuration.

CentOS

Legacy boot mode

grub2-mkconfig -o /boot/grub2/grub.cfg

UEFI boot mode

grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

Debian

Legacy boot mode

grub-mkconfig -o /boot/grub2/grub.cfg

UEFI boot mode

grub-mkconfig -o /boot/efi/EFI/debian/grub.cfg

Ubuntu

Legacy boot mode

grub-mkconfig -o /boot/grub2/grub.cfg

UEFI boot mode

grub-mkconfig -o /boot/efi/EFI/ubuntu/grub.cfg

Execute the following command to restart the instance for the configuration to take effect.

reboot

Step 2: Verify configuration

Execute the following command to view startup parameter information.

cat /proc/cmdline | grep "timeout"

As shown in the following figure, the timeout period for both instance I/O operations and Admin operations is 4294967295 seconds.

Execute the following command to view parameter information of the NVMe module.

cat /sys/module/nvme_core/parameters/io_timeout
cat /sys/module/nvme_core/parameters/admin_timeout

As shown in the following figure, the timeout period for both instance I/O operations and Admin operations is 4294967295 seconds.

Last updated: 2025.12.24 18:51:57