Docs

Torch Log Service

Timed SQL Analysis

Create schedule SQL analysis and store data in topic

TLS supports creating SQL analysis tasks in the Console. These tasks regularly query and analyze the source topic, and then save the processed log data to a target topic, or convert data into time series metrics for storage in the Prometheus workspace.

This topic describes how to create a scheduled SQL analysis task and save data to a targetlog topic.

Prerequisites

The log data has been successfully collected in the source log topic. For detailed instructions, see Overview.

A target log topic has been created to store the processed log data. For detailed instructions, see Log topic.

Both the source and target log topics are indexed. For detailed instructions, see Configure index.

Before using the schedule SQL feature for the first time, authorize cloud resource access. This grants schedule SQL analysis tasks permission to access and write data to the specified log topic. After authorization and successfully starting the schedule SQL analysis task, it uses the ServiceRoleForTLS role to write calculation results to the specified log topic.

Notice

When you create a schedule SQL analysis task for the first time in the TLS console, the system will guide you through the authorization process. We recommend using a Volcanoengine account for authorization. For details, see cross-service access authorization.

Steps

Select the region where log service is located from the top navigation bar.

On the left navigation bar, select Data Processing > Schedule SQL Analysis.

On the schedule SQL analysis task list page, click Create schedule SQL analysis task.

On the schedule SQL task page creation page, complete the following configurations.

Enter the basic settings for the task and click Next step: SQL analysis rule configuration.

Configuration	Description
Task name	Task name of the schedule SQL analysis task.
Description	A brief description of the schedule SQL analysis task.
Source project	The project where the raw logs for schedule SQL analysis are stored.
Source topic	The topic where the raw logs for schedule SQL analysis are stored.
Target topic	The target region, project, and topic where processed log data is stored after executing the schedule SQL analysis task. Target type: select TLS log topic. Target Region: Select the region where log data will be stored. Project: Select the project where log data will be stored. Log topic: Select the log topic where log data will be stored, ensuring that the topic has been indexed.
Task status	Determines whether to start the schedule SQL analysis task immediately. Enable (default): The schedule SQL analysis task starts immediately after configuration. Disenable: After configuration, the schedule SQL analysis task remains paused and requires manual start.

Enter the SQL analysis rule configuration, and then click Submit.

Configuration	Description
Start time	The schedule SQL analysis task’s start time, i.e., the time when the first instance is scheduled. After starting, TLS creates instances and runs schedule SQL analysis on time according to specified rules.
End time	The end time for scheduling the SQL analysis task. After the task ends, TLS will not create instances or perform scheduled SQL analysis. However, the task remains running until it is paused manually.
Schedule period	The schedule period for scheduled SQL analysis tasks. It determines the timing for each instance. The schedule period supports specifying time intervals or fixed trigger times. It is recommended to set the schedule period to be greater than or equal to the length of the query time window. Otherwise, analysis might be delayed and the query results may be inaccurate. For example, if the schedule period is every 10 minutes, the maximum length of the SQL time window is 10 minutes. You can select any of the following configurations: Minute: Runs at specified minute intervals, ranging from 1 to 1440 minutes. Day: Runs at a specific time every day. Cron: Uses a Cron expression to specify the schedule time. The minimum interval is one minute, and the maximum is 30 days in a 24-hour format. For example, 0 18 * * * schedules a SQL analysis task every day at 18:00.
Time zone selection	You need to select a time zone when the scheduling cycle is set to Cron. For a list of common zones, see Select a time zone.
SQL time window	When a scheduled SQL analysis task runs, the time range for log query and analysis uses a left-closed, right-open format. The maximum SQL time window is 31 days, and the minimum is 1 minute. If the SQL time window is set to more than 24 hours, execution may fail or results may be inaccurate due to excessive data volume. For SQL time window syntax, see SQL time window syntax. Notice Configuration example: Start time: 2023-8-14 10:00 Scheduling interval: Execute once every 60 minutes SQL time window: [@h-1h,@h) Execution delay: 10 seconds The first instance is created at 2023-8-14 10:00:00 and starts execution at 10:00:10. This instance analyzes log data from the period [09:00:00~10:00:00). The second instance is created at 2023-8-14 11:00:00 and starts execution at 11:00:10. This instance analyzes log data from the period [10:00:00~11:00:00).
SQL	The scheduled SQL analysis task executes query statements periodically. These statements must follow the query and analysis syntax of TLS. For detailed syntax instructions, see Query overview and Analysis overview.
Task timeout	If the scheduled SQL task exceeds the maximum timeout or maximum retry attempts, its status changes to “Failed” and the task ends. You can receive alerts and retry manually. Maximum duration: 60–1800 seconds. Maximum attempts: 1–20 times.
Execution delay	The delay for each schedule execution. Its supported settings range from 0 to 120 seconds. When the log data volume is large or the collection task is busy, writing data to the log topic may be delayed. In this case, specify an execution delay to ensure data completeness during query and analysis.

Last updated: 2026.03.16 14:05:02