Docs

Torch Log Service

LogBackFlow

Create data backflow task

The data backflow task will automatically create a data transformation task and a log delivery task (if the target type is set to deliver logs to TOS), helping you export eligible logs.

This topic explains how to create a data backflow task.

Prerequisites

The source project and log topic have been created, with key-value indexing configured.

The target project and log topic have been created, with indexing configured.

If TOS is chosen as the target for log delivery, the following conditions must be met:

Torch Object Storage(TOS) has been enabled, and a bucket has been created in the same region as the specified log topic. For more details, see Creating bucket.

TLS needs to assume the ServiceRoleForTLS role to enable cross-service access to other cloud resources. You must grant TLS permission to access and write to TOS resources.

If you have not yet created the ServiceRoleForTLS role, TLS will guide you through the process when you create the delivery configuration. We recommend authorizing with the Volcanoengine account. For details, see Cross-service access authorization.

Create data backflow task

Select the region where log service is located from the top navigation bar.

In the left navigation bar, select AgentLoop > LogBack Flow.

On the LogBack Flow page, click Create LogBack task.

On the LogBack task creation page, follow the prompt instructions to complete the configuration.

Enter the basic configuration.

Configuration	Description
Task name	Task name for the data backflow task. This name will automatically be used as the name for the scheduled SQL task.
Description	A brief description of the data backflow task.

Configure the data source.

Configuration	Description
Data source type	Select the data source type for the data backflow task: currently, only Torch Log Service TLS is supported.
Log project	The project containing the original logs for the data backflow task.
Log topic	The topic containing the original logs for the data backflow task.

Configure filter rules.

Configuration	Description
Backflow field	Backflow fields are sourced from key-value fields in the data source index. Custom fields are supported to add. You can use the original field name or set an alias (aliases can not be duplicated) as the exported field name. You must set field aliases in the following scenarios: JSON format fields Reserved fields in TLS
Condition configuration	You can filter results by setting conditions for callback fields and support multiple AND relationship filters.

Write data processing rules.

The system automatically generates AgentLoop transformation rules. It is not recommended to edit these rules directly.

You can add custom data transformation rules in the input box according to the onscreen instructions. TLS supports a variety of DSL syntax functions. On the transformation functions tab, you can click a function name to view syntax descriptions and examples, and insert specified syntax with one click to quickly write transformation rules. For the list of supported syntax functions, see Functions overview.

Test data processing rules.

Follow the page instructions to test by using log data from the log topic or specified test data. Click Execution preview to test processing rules.

Configure scheduling rules.

Configure scheduling rules for LogBack Flow tasks.

Configuration	Description
Start time	The scheduling start time of the log backflow task, i.e., the time when the first instance is scheduled to start.
End time	The scheduling end time of the log backflow task. The end time can not be earlier than the start time or the current time.

Set the target.

Configuration	Description
Target Region	The target region where the log of data backflow is located.
Log project	The target project where the log of data backflow is stored.
Log topic	The target topic where the log of data backflow is stored. The target topic can not be the same as the source log topic.
Other targets	Supports delivery to AgentLoop evaluation set or TOS. After enabling, you need to configure delivery target information.

When delivering to AgentLoop review set, the following parameters needs to be configured:

Configuration	Description
Review set method	Supports choosing an existing review set or creating a new one
Target review set (Name)	Select a target review set or enter a name to create a new one
Field mapping to review set	Select the corresponding review columns for the original fields.

When delivering to TOS, the following parameters needs to be configured:

Delivery target	Description
Bucket	The data will be delivered to the specified bucket name in TOS.
Select file path	Select or manually enter the file prefix (folder path) or the complete file path.
Shard format	The TOS Bucket directory will be generated dynamically based on the creation time of the delivering task. Cannot start with a forward slash (/). The default value is %Y/%m/%d/%H/%M, such as 2022/06/30/14/00.
Compression mode	Compression mode for the data, supporting No compression, snappy, gzip, lz4.
Delivering format	Data delivering format, supporting CSV, JSON, and JSONL.

Click Submit.

Analyze data backflow logs

After the LogBackFlow task runs successfully and the data transformation task is completed, you can view and analyze the data backflow logs at the target destination.

On the top navigation bar, select the region where the log service is located.

On the left navigation pane, select Resource management > Project.

Click the name of the target project to enter the project details page.

Click the target log topic name or click Actions and select query and analysis to access the log query and analysis page.

Enter your query statement manually or use the AI assistant to analyze log data returned. For details, see Log analysis.

Notice

The target log topic must enable index configuration.

View target project and log topic in data transformation tasks.

In the data backflow task list, click the data transformation task name in the data transformation task name/ID column to enter the details page, where you can view the target project, target log topic name, and data transformation rules.

Next steps

View and edit data return tasks. For details, see Manage data backflow task.

Last updated: 2026.07.29 18:09:36