Docs

Elastic Compute Service

Monitoring

Querying and responding to "SystemFailure.Reboot" events

This topic describes how to query and respond to "SystemFailure.Reboot" events in the console and through OpenAPI. This helps you quickly troubleshoot issues and run your cloud products stably.System failure instance restart

tip

Only instances of Big Data, Local SSD, and HPC-optimized instance type families can trigger this event. For details, refer to the instance type list.

Applicable scenario

When the host machine (host) of an ECS instance crashes due to a failure, it may be necessary to restart the instance to restore normal operations of the instance. To help you quickly resolve the failure, the platform sends you event-based notifications and also offers you comprehensive automatic O&M methods.

Scenario 1: Keep instance stopped

This scenario applies when you do not want the instance to be restarted or redeployed to another host machine when an event is triggered.

Operation processes

After an ECS instance triggers this event, it will not be restarted or redeployed. Meanwhile, an event indicating the instance is successfully stopped due to a system failure is generated to notify you.

tip

For instances in the Stopped status, billing continues.

Procedure

In the top menu bar, select the project and region of the target instance.

In the left-side navigation pane, select "Instances & Images > Images". The Instances page appears.

Select "... > Operations and troubleshooting > Instance maintenance property" in the "Actions" column on the right side of the target instance.

In the pop-up window, check "Maintain instance shutdown status". For detailed automatic O&M configurations, see Configuring automatic O&M for system events.

Scenario 2: Authorize redeploying instance after a restart fails

This scenario applies when you want the instance to be automatically restarted to resolve failure, and to grant permission first before your instance can be redeployed to another host machine if the restart fails.

Operation processes

After triggering this event, the ECS instance is automatically restarted. If the restart succeeds, the instance restores its normal operation, and an event indicating the instance is restarted successfully after a system failure is generated. If the restart fails, two types of events are generated: one indicating the instance fails to restart after a system failure, and another indicating that the platform is inquiring about the response to instance redeployment (triggered by a system failure). You can authorize the platform to deploy the instance to a new host machine. If no authorization is made, the instance remains in the Stopped status.

tip

For instances in the Stopped status, billing continues.

Procedure

In the top menu bar, select the project and region of the target instance.

In the left-side navigation pane, select "Instances & Images > Images". The Instances page appears.

Select "... > Operations and troubleshooting > Instance maintenance property" in the "Actions" column on the right side of the target instance.

In the pop-up window, check "Automatic Restart Instances". For detailed automatic O&M configurations, see Configuring automatic O&M for system events.

When you receive an event indicating the instance fails to restart after a system failure, and an event indicating that the platform is inquiring about the response to instance redeployment (triggered by a system failure), please authorize VolcanoEngine to redeploy your instance to a new host machine.

Respond in the console

In the top menu bar, select the region where your business is located.

In the left-side navigation pane, select "Maintenance & Monitoring > System Events".

Click the "Unexpected operation and maintenance events" tab, and filter for the target event indicating that the platform is inquiring about the response to instance redeployment (triggered by a system failure) by event ID or time.

In the "Actions" column of the target event, click the "Authorize maintenance" button to authorize.

After successful authorization, the event status changes to Executing. After being authorized, the platform begins deployment, which is expected to take 5 minutes.

After successful redeployment, the event status changes to Succeeded.

Respond through OpenAPI

In this section, the Java language interface is used as an example. The following is for your reference on calling OpenAPI interfaces in sequence to query and respond to ECS instance events.

tip

Relevant interfaces in this section are called through the SDK. Please import the SDK into your project. For details, see SDK overview.

If you do not want SDK to be introduced, you can create HTTP/HTTPS requests manually. For details, see Calling methods.

Call the DescribeSystemEvents interface to query information about events triggered by ECS Instances.

Sample request:

public void queryInstanceEvents(){
//Specify your AK
String ak = "Your AK";
//Specify your SK
String sk = "Your SK";
//Specify the region to be accessed. In this example, China (Beijing) is used.
String region = "cn-beijing";

//apiclient initialization
ApiClient apiClient = new ApiClient()
.setCredentials(Credentials.getCredentials(ak, sk))
.setRegion(region);

EcsApi ecsApi = new EcsApi(apiClient);

DescribeSystemEventsRequest describeSystemEventsRequest = new DescribeSystemEventsRequest();

//Specify the ID of the instance that you want to query
describeSystemEventsRequest.setResourceIds(Arrays.asList("<A Instance ID>"));
//Filter the event by the time the event is generated
describeSystemEventsRequest.setCreatedAtStart("2023-05-20T01:29:38+08:00");
//Filter the event by the time the event ends
describeSystemEventsRequest.setCreatedAtEnd("2023-05-25T18:29:38+08:00");

try {
DescribeSystemEventsResponse describeSystemEventsResponse = ecsApi.describeSystemEvents(describeSystemEventsRequest);
System.out.println(describeSystemEventsResponse);

} catch (ApiException e) {
System.out.println(e.getResponseBody());

}
}

Sample response:

class DescribeSystemEventsResponse {
nextToken:
systemEvents: [class SystemEventForDescribeSystemEventsOutput {
createdAt: 2023-05-25T14:15:21+08:00
id: e-ycfroor651gh9zxxx
operatedEndAt: 2023-05-29T11:58:30+08:00
operatedStartAt: 2023-05-28T23:58:30+08:00
resourceId: <A Instance ID>
status: Inquiring
type: SystemFailure_Redeploy
updatedAt: 2023-05-26T11:58:30+08:00
}]
}

Call the DescribeEventTypes interface to query whether events triggered by an instance need a response.

Sample request:

public void queryEventType(){
//Specify your AK
String ak = "Your AK";
//Specify your SK
String sk = "Your SK";
//Specify the region to be accessed. In this example, China (Beijing) is used.
String region = "cn-beijing";

//apiclient initialization
ApiClient apiClient = new ApiClient()
.setCredentials(Credentials.getCredentials(ak, sk))
.setRegion(region);

EcsApi ecsApi = new EcsApi(apiClient);

DescribeEventTypesRequest describeEventTypesRequest = new DescribeEventTypesRequest();

//Specify the type of the event that you want to query
describeEventTypesRequest.setTypes(Arrays.asList("SystemFailure_Redeploy"));

//Filter the event that needs a response
describeEventTypesRequest.setResponseRequired(true);

try {
DescribeEventTypesResponse describeEventTypesResponse = ecsApi.describeEventTypes(describeEventTypesRequest);
System.out.println(describeEventTypesResponse);
} catch (ApiException e) {
System.out.println(e.getResponseBody());
}
}

Sample response:

In this practice, a response is required to the event triggered by instance A, where the platform is inquiring about authorizing instance redeployment due to a system failure.

class DescribeEventTypesResponse {
eventTypes: [class EventTypeForDescribeEventTypesOutput {
kind: UnexpectedOperations
responseRequired: true
status: [Inquiring, Executing, Succeeded, Failed, Canceled]
title: SystemFailure.Redeploy
type: SystemFailure_Redeploy
}]
nextToken:
}

Call the UpdateSystemEvents interface to respond to the event where the platform is inquiring about authorizing instance redeployment due to a system failure. After the response, the platform is authorized to redeploy instances and resolve faults.

Sample request:

public void updateEvents(){
//Specify your AK
String ak = "Your AK";
//Specify your SK
String sk = "Your SK";
//Specify the region to be accessed. In this example, China (Beijing) is used.
String region = "cn-beijing";

//apiclient initialization
ApiClient apiClient = new ApiClient()
.setCredentials(Credentials.getCredentials(ak, sk))
.setRegion(region);

EcsApi ecsApi = new EcsApi(apiClient);

UpdateSystemEventsRequest updateSystemEventsRequest = new UpdateSystemEventsRequest();

//Pass in the ID of the event that needs a response
updateSystemEventsRequest.setEventIds(Arrays.asList("e-ycfu1fit1f8hcczyxxx"));
//Respond to the event
updateSystemEventsRequest.setStatus("Executing");
//Specify the time window for redeploying the instance

try {
UpdateSystemEventsResponse updateSystemEventsResponse = ecsApi.updateSystemEvents(updateSystemEventsRequest);
System.out.println(updateSystemEventsResponse);
} catch (ApiException e) {
System.out.println(e.getResponseBody());
}
}

Sample response:

class UpdateSystemEventsResponse {
operationDetails: [class OperationDetailForUpdateSystemEventsOutput {
error: null
eventId: e-ycfu1fit1f8hcczxxx
}]
}

Call the DescribeSystemEvents interface, pass in the ID of a responded event, and query information such as the current status of this event.

Sample request:

public void queryInstanceEvents(){
//Specify your AK
String ak = "Your AK";
//Specify your SK
String sk = "Your SK";
//Specify the region to be accessed. In this example, China (Beijing) is used.
String region = "cn-beijing";

//apiclient initialization
ApiClient apiClient = new ApiClient()
.setCredentials(Credentials.getCredentials(ak, sk))
.setRegion(region);

EcsApi ecsApi = new EcsApi(apiClient);

DescribeSystemEventsRequest describeSystemEventsRequest = new DescribeSystemEventsRequest();
//Specify the ID of the event to be queried
describeSystemEventsRequest.setEventIds(Arrays.asList("e-ycfu1fit1f8hcczyxxx"));

try {
DescribeSystemEventsResponse describeSystemEventsResponse = ecsApi.describeSystemEvents(describeSystemEventsRequest);
System.out.println(describeSystemEventsResponse);
} catch (ApiException e) {
System.out.println(e.getResponseBody());
}
}

Sample response:

Response example of an event in Executing status:

class DescribeSystemEventsResponse {
nextToken:
systemEvents: [class SystemEventForDescribeSystemEventsOutput {
createdAt: 2023-05-26T11:58:30+08:00
id: e-ycfu1fit1f8hcczyxxx
operatedEndAt: 2023-05-29T11:58:30+08:00
operatedStartAt: 2023-05-28T23:58:30+08:00
resourceId: <A Instance ID>
status: Executing
type: SystemFailure_Redeploy
updatedAt: 2023-05-26T14:19:19+08:00
}]
}

Response example of an event in Succeeded status:

class DescribeSystemEventsResponse {
nextToken:
systemEvents: [class SystemEventForDescribeSystemEventsOutput {
createdAt: 2023-05-26T11:58:30+08:00
id: e-ycfu1fit1f8hcczyxxx
operatedEndAt: 2023-05-29T11:58:30+08:00
operatedStartAt: 2023-05-28T23:58:30+08:00
resourceId: <A Instance ID>
status: Succeeded
type: SystemFailure_Redeploy
updatedAt: 2023-05-28T23:58:30+08:00
}]
}

Scenario 3: Automatically redeploy instance

This scenario applies when you want the instance to be automatically restarted to resolve the failure, and to be automatically redeployed to another host machine if the restart fails.

Operation processes

After an ECS instance triggers this event, it is automatically restarted. If the restart succeeds, the instance restores its normal operation and is not redeployed. If the restart fails, two types of events are generated: one indicating the instance fails to be restarted after a system failure, and another indicating the instance is being redeployed due to a system failure. After receiving this notification, the O&M personnel will redeploy your instance to a new host machine, which is expected to take 5 minutes.

Procedure

In the top menu bar, select the project and region of the target instance.

In the left-side navigation pane, select "Instances & Images > Images". The Instances page appears.

Select "... > Operations and troubleshooting > Instance maintenance property" in the "Actions" column on the right side of the target instance.

In the pop-up window, check "Automatically Redeploy (Default)". For detailed automatic O&M configurations, see Configuring automatic O&M for system events.

When the ECS instance is redeployed successfully after triggering this event, an event indicating the instance is successfully redeployed due to a system failure is generated to synchronize the O&M result with you.

Last updated: 2025.11.05 10:29:11