The Flink console provides a job details page where you can view basic job information, configuration details, monitoring metrics, snapshots, logs, and alert configurations.

Accessing the feature #

1. Log in to the Flink console.
2. Select the target region from the top menu bar.
3. From the left navigation pane, select Project Management, and then click the target project card to enter the project details page.
4. From the project's left navigation pane, select Job O&M > Job Management.
5. On the Jobs tab, filter for the target job and click its name.

Details overview #

• Basic Information: displays basic job information, such as the name, status, type, resource pool, priority, and engine version, and provides a quick link to the Web UI.
• Network Information: displays the subnet and security group information for the resource pool where the current job is running.
• Historical Instances: lists all historical job instances, allowing you to view the configuration details, monitoring metric curves, and logs for each instance.

Configuration Information #

Displays the job's current configuration details, including the following information:

• SQL Configuration: displays the job's SQL statement for SQL jobs only.
• Job Configuration: displays the JAR URI, main class, and the main function's args[] parameters for JAR jobs. Displays information such as the code file and Entrypoint for Ray jobs. Displays information such as the Python File URI and Entry Point Main Arguments for Python jobs.
• Job Resource Configuration: displays the job's Flink parameter configuration.
• Checkpoint Configuration: displays the checkpoint configuration of the task.
• Task Restart Strategy Configuration: displays the restart strategy and rules for the task.
• Task Failure Retry Configuration: displays the rules for retrying failed tasks, including the maximum number of retries and the retry interval.
• Custom Parameters: displays the custom parameters configured for the task.
• Auto-tuning:
• Dependency Files: displays the dependency files selected for the task.

Runtime Events #

Logs key events throughout the entire task runtime and displays critical information. Helps operations personnel troubleshoot, analyze, and identify the cause of errors.
Provides direct access to the instance's configuration details.

Data Curves #

Flink is integrated with Cloud Monitor to provide monitoring and alerting capabilities for Flink jobs.
The Data Curves tab on the Flink job details page displays the monitoring metric data curves for the current job. You can view key metrics across different dimensions as needed, including Overview, Network & IO, JVM, Schedule Info, Watermark, Checkpoint View, Checkpoint Timer, Checkpoint Operator Performance, Resource, and Kafka. Analyzing these metrics helps you quickly identify any job abnormalities.

Auto-tuning #

The Flink platform now supports auto-tuning for jobs. Once enabled, the system will auto-scale the job based on the configured time rules.

1. In the left navigation pane, go to Job O&M > Job Management.
2. On the Flink Jobs tab, filter for and click the target job name.
3. On the Auto-tuning tab, click New Rule, configure the tuning parameters, and click OK.
   If auto-tuning is enabled and you need to modify the tuning rules, click Edit Rules.
   
4. After a rule is created, click the enable button to activate the corresponding tuning task. The task will be scheduled once the rule's status is Enabled.
   
5. Adjust priority: Drag and drop priorities to change the scheduling order of tuning rules. If there is a time overlap, the rule with the higher priority is executed first.
   
6. The parameters are described as follows:

For more information on the limitations of auto-tuning, refer to Enable Auto-Tuning

#

Snapshots #

Displays task snapshots of Flink tasks created manually or by the system within the last 5 days.
Snapshots are primarily used to restore tasks after a system failure, preventing data loss. For details on how to create and use snapshots, see 作业快照.

Logs #

On the Logs tab of the task details page, you can view the log details for the current task. The system retains logs for 7 days. You can view the task's historical logs during the log retention period.

Alerts #

Flink integrates with Cloud Monitor to provide monitoring and alerting for Flink tasks.
On the Alert Configuration tab of the Flink task details page, click Go to Cloud Monitor to Configure Alert Rules to create alert rules and notification methods for the current task. When a monitored metric reaches the alert threshold, the system triggers an alert event and sends a message to the alert contacts. This helps you promptly detect and handle task exceptions.
To learn how to create an alert policy, see 创建告警策略.

Intelligent Diagnosis (Beta) #

Note: The Intelligent Diagnosis feature is currently an invitation-only feature. To enable it, Technical Supportsubmit a ticket.

Flink Intelligent Diagnosis collects real-time job metrics, such as latency, throughput, backpressure, CPU/memory usage, GC, Checkpoint status, operator watermarks, and data skew. It combines a rule engine with AI large model analysis to automatically identify anomalies, bottlenecks, and potential risks in streaming jobs, and then generates actionable repair suggestions or automated actions.

On the Intelligent Diagnosis tab in Task Details, select Intelligent Diagnosis to run an automatic diagnosis. The main supported features are:

• Restart Diagnosis: Analyzes the causes of restarts and suggests avoidance methods based on task restart events and related logs.
• Data Processing Analysis: Provides graphical DAGs, critical paths, hotspot metrics, and anomaly trends to help business and data teams quickly understand issues.
• Task Configuration Analysis: Analyzes a task's resource usage based on its configuration and provides optimization suggestions.
• Resource Usage Analysis: Analyzes core resource usage, such as CPU and memory, for the resource pool, JobManager, and TaskManagers, and provides optimization suggestions.

AI Diagnosis: Provides analysis capabilities enhanced by large models. It performs a comprehensive diagnosis of a task's end-to-end information and offers risk root cause localization, diagnostic reports, and repair solutions. Users can also use interactive Q&A for a more in-depth technical analysis of complex path issues, hidden bottlenecks, and potential risks.

In the AI Diagnosis feature, click the Diagnose Now button to automatically run a diagnosis with the large model. AI Diagnosis performs a comprehensive diagnosis of your Flink job by combining real-time metrics, logs, and other information. It also generates a diagnostic report with optimization suggestions.
You can further communicate with the large model in the dialog box to get more information about the job and receive a more in-depth diagnosis.