Docs

Torch Log Service

Getting Started

Collect log data

Based on an example using single-line full-text mode, this topic introduces how to create a collection configuration for collecting VolcanoEngine Elastic Compute Service (ECS) logs through the TLS console.

Prerequisites

The log project and log topic are created.

An available host group is created.

Create collection configuration

Select log space

After the log topic is created, click Continue: Access log data in the popup window.

If the log topic has already been created, log in to the TLS console, select the region North China 2 (Beijing) in the top navigation bar, and in the left sidebar, select Common features > log collection.

On the LogCollector log collection tab, click Create collection configuration.

In the Select log space step, configure the following parameters, and then click Next step: Select host group.

Parameters	Example value	Description
Log project	project-a	The collected log data will be stored in this project.
Log topic	topic-a	The collected log data will be stored in the selected log topic.

Select host group

In the Select log space step, configure the following parameters.

Parameters	Example value	Description
Install LogCollector	/	In this example, set it to Volcano ECS. Manual installation of LogCollector is not required.
Select host group	Host-group-a	Select the host group to be collected. If no host group is available in the list, you need to create a host group first.

After confirming the parameters, click Next step: Collection rules.

Configure collection rules

Configure basic information.

Parameters	Example value	Description
Rule name	Single-line full-text collection	Name of the LogCollector collection configuration. Supports: click Import other collection rules, and select Log region, Project, and Collection configuration to import existing collection configurations into the current one. You only need to specify the collection rule name.
Container log collection	Disabled	By default, the container log collection is disabled. If you collect host group logs or collect container logs via Sidecar, keep this setting disabled.
Collection path	/var/log/access.log	Collection path refers to the directory and filename of the logs. LogCollector matches directories based on the directory part of the collection path and monitors log files in those directories that meet the rules. You can set up to 10 different collection paths. You can specify a complete directory and filename for the collection path, or use wildcards for fuzzy matching. When the wildcard ** is specified in the log collection path, the path matches multiple directory levels. In this case and only in this case, TLS monitors subdirectories up to 8 levels deep under the specified directory. TLS currently supports the following wildcards: asterisk , double asterisk , and question mark ?. Only one double asterisk * can be configured at most.
Collection configuration effectiveness policy	Can be overwritten by new configuration	Set collection configuration effectiveness policy. Can be overwritten by new configuration: When an updated collection configuration exists, log files will not be collected according to the current configuration but will be collected based on the latest collection configuration. Permanent effect: Even if updated collection configurations exist, log files will continue to be collected based on the current configuration. Permanent effect only applies to LogCollector versions 2.1.0 and later versions and may result in duplicate collection costs. For details, see Collect multiple log files or standard output simultaneously.
Collection path denylist	Do not enable	After configuring the collection path denylist, TLS skips the specified directories and files when collecting logs through LogCollector. After selecting Enable, set the denylist directories and files as prompted on the page.

Configure log format.

Parameters	Example value	Description
Collection policies	Increment	Select the log collection policy. Full: collect all logs from the beginning of the file, including historical logs. Increment: collect the tail logs based on the increment threshold, and collect newly added logs in real time. After triggering LogCollector log collection, you can not collect historical logs from this file by changing the collection policy to Full.
Enter increment threshold	10240 KiB	If you select increment collection strategy, you need to set the increment threshold. The increment threshold refers to the file size from the starting collection position to the end of the file, when the configuration first takes effect. If the file size is smaller than this value, collection starts from the beginning.
Collection mode	Single line full text	LogCollector log parsing mode. In single-line full text mode, \n is used as the end marker for a log entry. Each log line is encapsulated into the field __content__. The original log itself will no longer be structured.
Time field	Acquisition point in time	Set the log collection time field. Acquisition point in time: use the system time of the server where LogCollector runs as the log timestamp. Custom time: extract the time embedded in the original log as the log timestamp. Enter the time key name, time regex pattern, and conversion format.
Ignore collecting unupdated files	Do not enable	If enabled, LogCollector skips files whose log file update time falls outside your specified time range when collecting data.
Upload failed parsing logs	Enable	All logs that fail to parse are uploaded to TLS, using the configured failure log key name field as the key and the original log content as the value.
Failure log key name	LogParseFailed	The default failure log key name is LogParseFailed.

Configure plugin.

Parameters	Example value	Description
Plugin configuration	Do not enable	When collecting text logs via LogCollector, if business logs can not be parsed by regular collection modes like JSON due to complex structures or inconsistent format, use LogCollector plugin for post-collection processing. For more details, see Plugin overview.

Configure advanced settings.

Select advanced configurations based on your needs. If you have no special requirements, we recommend that you keep the default configuration. This example keeps the default configuration.

Parameters	Example value	Description
Filter	Do not enable	When enabled, you can configure filtering rules by using regular expressions. Only logs that fully match the regular expression will be collected and reported. This helps you filter log data that you need.
Upload hostname field	Do not enable	Add a field in the original log to record the hostname of the log source. The field name can be specified via hostname key name. It is defaulted to hostname.
Parse collection path	Do not enable	Extract fields from the collection path by using regular expressions and add them as metadata to the log data.
HashKey routing shard	Do not enable	Specify the HashKey to write data into the log shard that meets the range requirements.
Upload original log	Do not enable	Upload the original log as a single field to TLS
Upload constant fields	Do not enable	Pack the specified field’s key and value into each log entry
Upload host group label	Do not enable	Upload host group label information to TLS.
Extended configuration	Disabled	LogCollector extended configuration.

After confirming the collection configuration, click Next step: check index configuration.

Next steps

You need to configure the index before previewing the collected log data in real time on the console or conducting log query and analysis. For more details, see Configure indexes.

Last updated: 2026.04.27 16:13:30