Torch Log Service
Torch Log Service
Torch Log Service
Document
Product News
Product Overview
Product Billing
User Guide
TLS Agent
AgentLoop
Resource management
Host group
Data collection
LogCollector collection
Introduction to LogCollector
Installation and upgrade
Collect host logs
Collect container logs
LogCollector plugins
Manage LogCollector
general reference
Data import
Cloud product log collection
TOS log collection
DBW log collection
VPN Log collection
VKE log collection
CLB log collection
CloudTrail log collection
CFW log collection
NTA log collection
ALB log collection
APMPlus log collection
veFaaS log collection
VPC log collection
veImageX log collection
TrafficRoute DNS log collection
DDoS log collection
WAF log collection
DCDN log collection
Multi-cloud security platform log collection
APIG log collection
Machine learning task log collection
CR log collection
Volcano Ark application lab log collection
FileNAS log collection
LLM Application Firewall Service log collection
vePFS log collection
Cloud security center log collection
EFS log collection
Data storage
Query and analysis
Configure log indexes
Query log
Analyze logs
Query and analyze logs using third-party tools
Associate external data source for log query
SQL function
SQL syntax
Visualization
Query and analysis charts
Summaries of query and analysis charts
Create dashboard
Use dashboard
Add dashboard filters and variables
Use other systems to analyze logs
Log application
Log applications for product observability
Log applications for large models observability
Coding Agent log app
ModelProvider log app
No-coding Agent log app
Log applications for basic service monitoring
Data processing
Data transformation
Syntax and functions
Global operation function
expression function
Data processing best practices
general reference
Data write processor
Data consumer processor
Data consumption and shipping
Ship logs to TOS
Ship logs to Kafka
Ship logs to Splunk
Alarm
Alarm policy
Alarm notification content
Alarm notification content template
alarm notification channel
Tracing service
Monitor Log Service
API reference
Project management API
Topic management API
Shard management API
Index management API
Log management API
Host group management API
Collection configuration management API
Data import API
Data processor API
Data transformation API
Data backflow API
SQL analysis API
Alarm management API
Consumer group management API
Kafka protocol consumption API
Tag management API
Trace storage API
Account management API
SDK reference
Go SDK
Java SDK
Python SDK
iOS SDK
C++ SDK
CLI reference
Best practices
Data collection
Query and analysis
permission management
Consumption and delivery
Visualization
Alarm
FAQs
Billing and cost FAQ
Data collection FAQ
Data storage FAQ
Index configuration FAQ
Query and analysis FAQ
Scheduled SQL tasks FAQ
Alarm FAQ
- Documentation
Torch Log ServiceUser GuideData processingTimed SQL AnalysisPrinciples of work
Copy page
Download PDFTimed SQL Analysis
Principles of work
Copy pageDownload PDFPrinciples of work
This topic describes the implementation process, scheduling examples, and basic concepts of scheduled SQL analysis tasks.
How it works
Process overview:
- When creating a scheduled SQL analysis task, users must provide basic configuration and configure SQL analysis rules.
- The backend service creates scheduling instances based on the configured time and cycle, and then executes the instances on schedule.
- If an instance is delayed, subsequent instances may also be delayed; in this case, the system will catch up, gradually reducing the delay until tasks run on time again.
- Each instance uses the configured query statement to perform query and analysis on the specified log topic, and writes the results into the designated log topic.
Scheduling example
For example, if you set the scheduling start time to 10:00:00 on August 17, 2023, with an execution cycle of one hour, an SQL time window of [@h-1h,@h), and a 30-second delay for execution, then: the first instance is scheduled at 11:00:00, runs at 11:00:30, and queries log data within the range [10:00, 11:00).
Basic concepts
terminologies | Description |
Scheduled SQL analysis task | A collection of scheduled SQL analysis rules, including basic configurations such as source and target log topics, scheduling cycles, and SQL analysis rule settings like SQL time windows. |
Instance | After a scheduled SQL analysis task starts, TLS creates instances on time based on the specified scheduling cycle. Each instance runs scheduled SQL analysis on the source log topic and writes the computed results to the specified log topic. Notice
|
Scheduling cycle | The scheduling cycle for scheduled SQL analysis tasks determines the scheduled time for each instance. |
Scheduled time | By default, the time when the scheduled instance begins execution is determined by the scheduling cycle and is not affected by previous instance timeouts, delays, or reruns. If the instance does not meet execution conditions at the scheduled time or delayed execution is set, execution will be postponed. |
Execution time | The actual execution time of the instance, i.e., the time when the scheduled SQL analysis task instance starts running. By default, the instance runs according to the scheduled time. If delayed execution is set or the previous instance has not finished and other execution conditions are not met, this time will be later than the scheduled time. |
SQL time window | When running a scheduled SQL analysis task, the time range for log query and analysis uses a left-closed, right-open format. After specifying the SQL time window, TLS initiates scheduled SQL analysis tasks on time based on the defined scheduling cycle and analyzes log data within the designated SQL time window. |
Delayed execution | Delay time for each scheduled task. When log data volumes are high and collection tasks are busy, writing data to log topics may experience some delay. In this scenario, you can specify a delay time to ensure data integrity during query and analysis. |
Last updated: 2026.03.16 14:05:02
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
