You need to enable JavaScript to run this app.
E-MapReduce

E-MapReduce

Copy page
Download PDF
Logs, monitoring, and alerts
Practices for configuring cluster monitoring and alerting
Copy page
Download PDF
Practices for configuring cluster monitoring and alerting

Configuring clustersMonitoring and alertsenables users to monitor the operational status and resource status of clusters in real time, promptly detect and handle anomalies, and ensure business stability. This article provides several types of basic Monitoring and alertsconfigurations and recommends applying these basic configurations.

Configure cloud server resource alerts
  1. Log in to the Cloud Monitoring Console . On the Alert center>Alert Template>Preset Alert Template page, under Resource Type , select Elastic Compute>Cloud Server to filter templates. For detailed instructions on creating and using alert templates, see Create Alert Template .
    Image
  2. Click the Cloud Server Preset Alert Template name to enter the template details page and view information such as alert trigger conditions, alert levels, and associated alert policies.
    Image
  3. On the right side of the Cloud Server Preset Alert Template details page, click Enable, or return to the Preset Alert Template page and click the Apply with One Click button. The system will automatically generate and enable alert policies, applying them to all cloud server resources under this Volcano Engine account.

    tip

    After disabling the alert template, all alert policies generated by the preset alert template will be deleted accordingly.

Configure EMR service resource alerts

On the Alert center>Alert Template>Preset Alert Template page, in Resource Type , select Data Platform>E-MapReduce to filter the templates. Refer to Configure cloud server resource alertsconfiguration steps to enable the E-MapReduce preset alert template.
Image

tip

Data Middle Platform>E-MapReduce also provides other preset alert templates, which can be enabled as needed.

Configure service status/operation exception event alerts
  1. Log in to the Cloud Monitoring Console and go to the Event Center>Event Rules>Create Event Rule page.
  2. Create service status/operation exception event alert rules,
    1. In the Resource Type dropdown, select E-MapReduce.
    2. In the Event Name dropdown, enter the keyword "status" to filter service status events and select event names. It is recommended to select all status events for services in use.
    3. In the Event Name dropdown, enter keywords such as "success" or "failure" for operation event keywords and select event names. You can configure based on high-frequency operations.
      For detailed guidance on creating event rules, see Create Event Rule .
      Image

Configure alerts for core service metrics

Core service metrics directly affect business stability. It is recommended to configure alert policies for these metrics to promptly detect and address anomalies.

  1. Log in to the EMR console, go to the EMR cluster details page>Alert center>Subscribe to event alerts, and navigate to the Cloud Monitoring alert policy page.
    Image

  2. On the Cloud Monitoring alert policy page, click Create alert policy to enter the creation page.

  3. Set Resource type to Data Middle Platform>E-MapReduce.
    Image
    Image

  4. Configure alert Dimension, metric types Trigger condition, and other alert policy details. For detailed parameter descriptions, see Create Alert Policy .
    The table below lists key service metrics and recommended alert policies. Please refer to these policies for configuration.

    Alert dimension

    Alert metric and Note

    Metric level

    Trigger condition

    HDFS

    DFS space used by HDFS

    Critical

    Alert when usage exceeds 90% of total capacity
    Total capacity query path: EMR Console > Cluster Details > Cluster Monitoring > HDFS > Overview > Storage Status.
    Image

    tip

    Recommendation: Increase the JVM heap size of the NameNode and clean up directories containing a large number of small files
    Alert sending interval recommendation: Check every 5 minutes; if it occurs twice within 30 minutes, trigger an alert.

    HDFS storage utilization

    Critical

    Alert when above 80%

    Number of lost HDFS blocks

    Critical

    Configure as needed. If lost blocks are not allowed, set alert for Missing block > 0.

    NameNode heap space usage ratio

    Critical

    Alert when above 80%

    DataNode old generation space utilization

    Critical

    Alert when above 95%

    tip

    Alert sending interval recommendation: The alert should persist for a certain period within a specified time frame.

    Yarn

    ResourceManager old generation space

    Critical

    Alert when above 95%

    NodeManager old generation space

    Critical

    Alert when above 95%

    Hive

    Number of open hiveserver2 connections

    Critical

    Set as needed

    HiveMetaStore heap memory utilization

    Critical

    Alert when above 95%

    HiveServer2 heap memory utilization

    Alert when above 95%

    Number of HiveServer2 query failures

    Set as needed

    Hbase

    HRegion full gc

    P2

    Monitor and troubleshoot if it occurs frequently.

    HRegion read rate

    P2

    Set as needed

    HRegion write rate

    P2

    Set as needed

    Presto/Trino Coordinator

    Current amount of HeapMemory used by JVM

    P1

    Alert for exceeding 95% of heap memory

    Presto/Trino Worker

    Current amount of HeapMemory used by JVM

    P1

    Alert for exceeding 95% of heap memory

    ZooKeeper

    Number of ZooKeeper znodes

    P2

    Set as needed

    ZookeeperOverview

    Total number of connections in ZooKeeper cluster

    P2

    Set as needed

    Kafka(KafkaConsumerGroup)

    Kafka Consumer Group Lag

    P1

    Set as needed

Last updated: 2026.07.08 12:00:08