Docs

Torch Log Service

Global operation function

Event handler function

This topic covers the syntax and structure of event handler functions, as well as related scenario examples.

e_output function

The e_output function outputs log data to the target topic with specified alias. The log topic alias refers to the target name specified when you create the transformation task.

Function syntax/format

The parameters are described below:

Parameters	Parameter type	Required	Default value	Value range	Description
topic	String	Yes	/	/	Log topic alias.

Function examples

Scenario: Based on the value of the is_output field in the log, this function determines whether to output the log to the log topic named target-1.

Processing rules:

c_if(v("is_output"), e_output("target-1"))

Log example

[
{
"is_output":1,
"content":"hello world"
},
{
"is_output":0,
"content":"test content"
}
]

Processing result:

[
{
"is_output":1,
"content":"hello world"
}
]

Tip

The first log is output to the log topic with the alias target-1, while the second log is not output.

e_auto_output function

Dynamically distributes logs to different log topics. For example, if the log field service is set to tos or kafka, you can specify the e_auto_output function to dynamically distribute logs to log topics named tos and kafka based on the value of service. If log topics tos and kafka do not exist, the system will create topics first.

Function syntax/format

The parameters are described below:

Notice

If a project or log topic with the same name already exists, the system will not create it again.

When dynamically creating a project or log topic using a field value, the field value must meet resource naming requirements. Otherwise, an error will occur when using the e_auto_output function. For details, see Resource naming limits.

Parameters	Parameter type	Required	Default value	Value range	Description
project	String	Yes	/	/	Sets the log project name.
topic	String	Yes	/	/	Sets the log topic name.
tags	String	No	/	/	Sets log topic tags in the format key:value. Multiple tags should be separated by commas. For example, if tags='a:b,c:d' is set, it means two tag pairs, a:b and c:d, are added to the log topic.
index	String	No	full_text	full_text	Sets index for the log topic. full_text: Enable Full-text index. Empty string: Disable Full-text index.

Function examples

Function example 1:

Scenario: This function extracts the value from the service field, then dynamically routes the log to the corresponding topic based on that value, and enables the full-text index for the topic. If the project or log topic does not exist, the system creates them based on the values specified in the function.

Processing rules:

e_auto_output(project='myproject',topic=v('service'))

Log example

// Log 1
{
"service": "tos",
"week": "Monday,Tuesday,Wednesday"
}
// Log 2
{
"service": "kafka",
"week": "Monday,Tuesday,Wednesday"
}

Processing result:

Route log 1 to the tos topic under the myproject project.

{
"service": "tos",
"week": "Monday,Tuesday,Wednesday"
}

Route log 2 to the kafka topic under the myproject project.

{
"service": "kafka",
"week": "Monday,Tuesday,Wednesday"
}

Function example 2:

Scenario: This function extracts values from the project-name field and topic-name field, and then distributes logs to the corresponding log topic based on these values; adds tags key1:value1 and key2:value2 to the log topic and enables the Full-text index. If the project or log topic does not exist, the system creates them based on the values specified in the function.

Processing rules:

e_auto_output(v('project-name'),v('topic-name'),tags='key1:value1,key2:value2')

Log example

// Log 1
{
"project-name": "myproject01",
"topic-name": "accesslog01",
"week": "Monday,Tuesday,Wednesday"
}
// Log 2
{
"project-name": "myproject02",
"topic-name": "accesslog02",
"week": "Monday,Tuesday,Wednesday"
}

Processing result:

Route log 1 to the myproject01 project under the log topic accesslog01. Then add tags key1:value1 and key2:value2 to this topic and enable the Full-text index.

{
"project-name": "myproject01",
"topic-name": "accesslog01",
"week": "Monday,Tuesday,Wednesday"
}

Route log 2 to the myproject02 project under the log topic accesslog02. Add tags key1:value1 and key2:value2 to this topic and enable the Full-text index.

{
"project-name": "myproject02",
"topic-name": "accesslog02",
"week": "Monday,Tuesday,Wednesday"
}

Function example 3:

Scenario: This function provides the project name and topic name from the project-name field and topic-name field; distributes logs to the corresponding log topic based on these values, and disables the full-text index for the log topic. If the project or log topic does not exist, the system creates them based on the values specified in the function.

Processing rules:

e_auto_output(v('project-name'),v('topic-name'),index='')

Log example

// Log 1
{
"project-name": "myproject01",
"topic-name": "accesslog01",
"week": "Monday,Tuesday,Wednesday"
}
// Log 2
{
"project-name": "myproject02",
"topic-name": "accesslog02",
"week": "Monday,Tuesday,Wednesday"
}

Processing result:

Route log 1 to the myproject01 project under the accesslog01 log topic, and disable the Full-text index for this log topic.

{
"project-name": "myproject01",
"topic-name": "accesslog01",
"week": "Monday,Tuesday,Wednesday"
}

Route log 2 to the myproject02 project under the accesslog02 log topic, and disable the Full-text index for this log topic.

{
"project-name": "myproject02",
"topic-name": "accesslog02",
"week": "Monday,Tuesday,Wednesday"
}

e_split function

Use a separator to split specific fields, resulting in multiple log entries.

Function syntax/format

Splitting rules:

Rule 1: If the jmes parameter is configured, convert the log field value into a JSON list and extract values using jmes for the next step. At this point, the sep and quote parameters are invalid. If no jmes parameter is set, the splitting will proceed by following Rule 2.

Rule 2: If the previous value is a list or a JSON-formatted list string, split by this list and end processing. The sep and quote parameters are invalid at this time. If it is not a JSON list or JSON list string, the splitting will proceed by following Rule 2.

Rule 3: Use sep, quote, and lstrip to parse the previous value with CSV, split by the resulting values, and end processing.

The parameters are described below:

Parameters	Parameter type	Required	Default value	Value range	Description
field	String	Yes	/	/	Name of the field to be split.
sep	String	No	，	Any single character	Separator
quote	String	No	"	Any single character	A reference symbol used to reference multiple values of paired characters.
lstrip	Boolean	No	True	/	Determines whether to remove leading spaces from the value. By default, it is True. This parameter takes effect only when the string is split.
jmes	String	No	None	/	Converts the field value to a JSON object, extract specific values using JMES, and then perform the split operation.
output	String	No	None	/	Sets a new field name. By default, the new field name will overwrite any existing field with the same name.

Function examples

Scenario: Splits into multiple logs based on the value of the city field.

Processing rules:

e_split("city")

Log example

{
"content":"hello world",
"city":"shanghai,hangzhou,chengdu"
}

Processing result:

[
{
"content":"hello world",
"city":"shanghai"
},
{
"content":"hello world",
"city":"hangzhou"
},
{
"content":"hello world",
"city":"chengdu"
}
]

e_drop function

Discards logs that meet the specified criteria.

Function syntax/format

The parameters are described below:

Parameters	Parameter type	Required	Default value	Value range	Description
condition	Boolean	No	True	/	Determines whether to discard or not.

Function examples

Scenario: Determines whether to discard a log based on the value of the city field.

Processing rules:

c_if(op_eq(v("city"), "shanghai"), e_drop())

Log example

[
{
"content":"hello world",
"city":"shanghai"
},
{
"content":"hello world",
"city":"hangzhou"
}
]

Processing result:

{
"content":"hello world",
"city":"hangzhou"
}

e_keep function

Keeps logs that meet the criteria; discards logs that do not meet the criteria.

Function syntax/format

The parameters are described below:

Parameters	Parameter type	Required	Default value	Value range	Description
condition	Boolean	No	True	/	Determines whether to keep or not.

Function examples

Scenario: Determines whether to keep the log based on the value of the city field.

Processing rules:

e_keep(op_eq(v("city"), "shanghai"))

Log example

[
{
"content":"hello world",
"city":"shanghai"
},
{
"content":"hello world",
"city":"hangzhou"
}
]

Processing result:

{
"content":"hello world",
"city":"shanghai"
}

Last updated: 2026.05.09 11:13:24