Docs

Torch Log Service

Data write processor

Write processor overview

This topic describes the features, usage restrictions, and other information about the write processor in TLS.

Notice

Write processor is in closed beta. If you have relevant business needs, contact your account manager for the allowlist.

Billing notes

Write processor is in closed beta and is free of charge for now. We will release billing notes in the upcoming announcement.

Features overview

The write processor in TLS is a powerful tool for real-time, lightweight data cleansing, transformation, and enrichment before data is written to the service. It is suitable for business scenarios that require unified formatting, filtering of invalid data, and protection of sensitive information before data is stored.

You can create a write processor directly in the TLS console, associate it with log topics, and use SQL-like DSL (Data Processing Language) to define rules for field string masking, encryption, renaming, conditional logic, flow control, and JSON field structuring.

Use DSL to define data processing rules. For details, see Data transformation overview.

Supported data processing functions. For details, see Functions overview.

Notice

The following functions in the function list are not supported:

Event handler functions: e_output function, e_auto_output function.

Resource function: res_rds_mysql function.

Features

Zero client overhead: All processing is completed on the server side and does not consume any client computing resources.

Centralized rule management: Manage the preprocessing logic for all data sources using unified DSL rules to ensure consistency.

Improve data quality: Effectively intercept "dirty data" and ensure data integrity and high value at the source.

Reduce costs and risks: Filter and mask data to lower the cost of storing invalid data and reduce the risk of sensitive information leaks.

Features comparison	Write processor	Consumer processor	Data transformation task
Processing stage	Data before writing to TLS	Data collected in TLS	Data collected in TLS
Applicability	Exists in the project and applies to log topics within the project. Cross-region, cross-account, and cross-project operations are not supported.	Exists in the project and applies to log topics within the project. Cross-region, cross-account, and cross-project operations are not supported.	An independent processing task that applies to the selected source log topic and target log topic Supports cross-region, cross-account, and cross-project usage.
Billing notes	Currently free of charge. The billing schedule will be announced later.	Currently free of charge. The billing schedule will be announced later.	Data transformation fees and cross-region transmission costs.
Processing rules	You can use built-in functions provided by TLS DSL to write processing rules. The following functions are not supported: e_output, e_auto_output, and res_rds_mysql.	You can use built-in functions provided by TLS DSL to write processing rules.	You can use built-in functions provided by TLS DSL to write processing rules.

Limits

Items	Specifications
Number of write processors	You can create up to 100 write processors in each project. For quota adjustments, see Apply for resource quota.
Association	Each log topic can associate only one ingestion processor. One ingestion processor can be associated with multiple log topics. Cross-project association of log topics is not supported.
Effective time	The ingestion processor takes effect within one minute after being created or edited.
Protocol compatibility	Compatible with all log data methods provided by TLS, including Logtail, various language SDKs/APIs, Web Tracking, and more.
DSL rules and performance impact	For details, see Data transformation overview.

Last updated: 2026.06.05 14:25:55