This topic describes how to authorize the replacement of a faulty local disk when a local volume fails.
warning
Please read carefully and follow this document, which is part of the ECS service rules provided to you. If you do not follow the instructions here, your business may suffer losses, and you acknowledge and agree to take full responsibility for the losses.
Scenarios
When you receive a DiskErrorDetected (disk exception) event, you can authorize VolcanoEngine to replace the faulty disk. This helps prevent more serious issues such as instance downtime caused by the disk failure.
Considerations
- This feature is currently not available for instances of the Elastic Bare Metal (EBM) type, Local SSD i2g, and HPC-optimized types (hpcpni2, ebmhpcpni2l, ebmhpcpni2, and ebmhpchfpni2). For details, refer to Instance types and specifications.
- Please follow the procedure to modify the local disk configuration first before you authorize O&M operations. Otherwise, the newly replaced local volume may not be recognized properly.
- There is a risk of data loss during replacement of the faulty volume. Please back up data in advance to prevent data loss. You can back up data using volumes, TOS, or other methods.
Procedure
Step 1: Obtain the serial number of the faulty hard disk
When a local volume mounted to an instance fails, VolcanoEngine will synchronize the serial number (SN) of the faulty hard disk with you. Therefore, you can locate the faulty hard disk for the instance accurately and easily.
You can call the DescribeSystemEvents interface, pass in the event ID of the hard disk exception, and obtain the SN of the faulty hard disk from the returned data. - If the instance with the faulty hard disk is a first-generation HDD instance (for example, Big Data Storage-optimized d1s), the value of the SystemEvents.ExtraInfo.Address parameter in the returned data is the SN of the faulty hard disk.
- If the instance with the faulty hard disk is a second-generation HDD instance (for example, Big Data d2s, Big Data d2c, and Big Data d2), or an instance of Local SSD type (for example, Local SSD i2), the value of the SystemEvents.ExtraInfo.Serial parameter in the returned data is the SN of the faulty hard disk.
Step 2: Modify the hard disk configuration
- Log in to the ECS instance where the faulty volume is mounted. For more information, refer to Login overview.
- Execute the following command to view the drive letter of the faulty local volume.
- First-generation HDD instance
- If your instance is a first-generation HDD instance (for example, Big Data Storage-optimized d1s), execute the following command to view the drive letter of the faulty hard disk.
Replace $SN$ with the actual SN of the hard disk.
- Second-generation HDD or Local SSD instance
- If your instance is a second-generation HDD instance (for example, Big Data d2s or Big Data d2c), or an instance of Local SSD type (for example, Local SSD i2), execute the following command to query the drive letter of the faulty hard disk.
Replace $SN$ with the actual SN of the hard disk.
- Execute the following command to view the file system and UUID of the faulty local volume.
Replace /dev/vda2 with the actual drive letter of the faulty volume obtained in the previous step.
- Execute the following command to edit the /etc/fstab file.
- Open the /etc/fstab file.
- Press the i key to enter edit mode, and add the nofail parameter after the target local volume in the file.
Only the nofail parameter needs to be added and no other content requires adjustment.
-
- Description of fstab file arguments
- Press Esc and enter :wq to save the change and exit.
- Execute the following command to unmount the faulty local volume.
Step 3: Authorize O&M
- In the left-side navigation pane, select "Maintenance & Monitoring > System Events" to go to the Events page.
- In the top menu bar, select the region and project of your business.
- On the Events page, you can search the target event based on the time range, event ID, or instance ID.
- Click the "Maintenance operation" button in the "Actions" column of the target event to go to the maintenance configuration page.
- Select "Replace the faulty volume" for "Operation and maintenance method".
- (Optional) Before you replace the faulty volume, if you need to prevent applications running on the instance from reading from and writing to the faulty volume to minimize data loss, authorize isolating the hard disk first. For more information, refer to Isolating the local volume (optional).
- Click the "Authorize maintenance" button, then click the "Confirm" button in the confirmation pop-up window to authorize VolcanoEngine O&M personnel to replace the faulty hard disk.
After the authorization, a DiskError.ReplaceDisk (Hard disk replacement due to exception) event will be generated. The event aims to inform you the progress of replacing the faulty disk.
- If an issue occurs during the faulty volume replacement, you will receive a DiskError.ReplaceDisk:Pending event notification. After you receive the notification, follow the instructions to restart the instance or submit a ticket to resolve the issue.
- Resolve the issue by restarting the instance
- As shown in the picture below, follow the instructions, click the "Restart instance" button, and restart the instance to resolve the issue.
- Resolve the issue by submitting a ticket
- As shown in the picture below, follow the instructions and submit a ticket to contact VolcanoEngine O&M personnel to resolve the issue.
Step 4: Mount the hard disk
- After the faulty volume is replaced, you will receive a DiskError.ReplaceDisk:Succeeded event notification. After receiving the notification, you need to log in to the instance to mount the local volume.
tip
- If the instance fails to identify the local volume after the faulty volume is replaced, submit a ticket for technical support.
- We recommend that you use UUID instead of device names (such as /dev/data1) for mounting.
- Execute the following command to view information about the unmounted data volume (the new local volume).
- Execute the following command to create a file system for the data volume to be mounted.
- Execute the following command to mount the data volume.
- Execute the following command to check whether the data volume is mounted.