Docs

Torch Log Service

LogCollector plugins

Plugin overview

When collecting text logs via LogCollector, if business logs can not be parsed by regular collection modes like JSON due to complex structures or inconsistent format, you can use LogCollector plugin for post-collection processing.

Features overview

LogCollector provides multiple text log collection modes, including separator mode, JSON mode, and full regular expression mode. It also supports multiple advanced configurations such as log filtering. It can meet the collection and processing needs for most typical scenarios. Some scenarios may be complex due to, for example, complex business log structure, the coexistence of logs in JSON mode and separator mode, and unstable log fields. Therefore, these logs can not be parsed and processed using one single mode. In these cases, use the LogCollector plugin for post-collection processing.

To set LogCollector rules for collecting text logs, you need to enable the plugin and fill in the plugin configuration in sequence for log processing. When TLS processes the collected log data, it runs the corresponding logic in the order that you specified in the plugin configuration. For detailed usage, see .

The LogCollector plugin module provides various plugins to cooperate with LogCollector data collection feature. With this combination, TLS enhances its post-collection processing capability in complex logging scenarios. For the list of currently supported plugins, see Plugin list. If you need to determine whether to run a plugin based on log content, you can add plugin conditions within the plugin configuration, such as checking whether a field value equals a fixed value or contains a specified string. For a list of supported conditions, see Execution conditions list.

Notice

After enabling plugin configuration, you can add one or more LogCollector plugins. LogCollector will execute plugins according to the specified order of LogCollector plugins. If any LogCollector plugin fails to execute, the entire LogCollector plugin process will fail.

After enabling plugin configuration, all advanced features that are currently available remain effective, including filter configuration, using collection time, upload parsing failure log, parse collection path, upload original log, upload constant field, and extended configuration.

LogCollector plugins usually need to parse the content of a particular field. When the collection mode is single-line or multi-line full text, the field name for single or multi-line full text logs is __content__. Therefore, the field name for the first LogCollector plugin configuration should be specified as __content__.

Container metadata fields can not be processed via plugins.

LogCollector plugin features are continuously updated. We recommend using the latest version of LogCollector for a comprehensive experience.

Plugin list

Category	LogCollector plugin	Description
Field extraction	delimiter plugin	The delimiter plugin enables LogCollector to extract fields via separator mode.
		fullregex plugin	The fullregex plugin enables LogCollector to extract fields using full regular expression mode.
		kv plugin	The kv plugin allows LogCollector to extract fields using key-value pairs.
Add fields	add_kvs plugin	The add_kvs plugin lets LogCollector add specified fields
JSON parsing	json plugin	The json plugin enables LogCollector to parse and expand fields in JSON mode.
Data masking	desensitize plugin	The desensitize plugin replaces specified log entries with their corresponding MD5 values to achieve data masking.
String replacement	replace plugin	The replace plugin replaces specified log content with a given string.
Time parsing	Time parsing plugin	The strp_time plugin parses time content from logs and records the parsed time as the log timestamp on the server.
Rename	Rename plugin	The rename plugin is used to rename field names.
Filter logs	Filetring log plugin	The drop_log plugin supports matching by log field names and regular expressions to filter out logs that do not meet specified criteria.
Drop log plugin	Drop log plugin	The drop_fields plugin removes unnecessary fields from logs.
JSON extraction plugin	JSON extraction plugin	The json_extract plugin extracts values from specified JSON content and adds them as a new JSON array to the log.
Field splitting and reassembly plugin	Field splitting and reassembly pluginplugin	The split_select plugin splits the value of a specified field. It combines selected parts into a new field and adds the new field to the log.
URL parsing plugin	URL parsing plugin	The url_parse plugin parses URL strings from specified fields and extracts each component of the URL to add to logs.
Keep field plugin	Field retentionplugin	The keep_fields plugin is used to retain specified fields and discard all others.

Last updated: 2025.12.05 15:26:14