Docs

Torch Log Service

Collect container logs

Collect container logs (Sidecar - Console method)

TLS supports collecting Kubernetes container text logs using the Sidecar method. This topic shows how to create a collection configuration in the console.

Background information

LogCollector collects container text logs using the Sidecar mode and uploads the collected log data along with container metadata to the server.

When collecting container text logs in Sidecar mode, LogCollector offers the following features:

Supports all functions that are available for LogCollector deployed on physical machines, including single-line, multi-line, separator, JSON, and complete regex collection modes.

LogCollector supports reserved fields defaulted in various collection configurations, and also supports uploading container metadata (such as Namespace, Pod name, Pod IP address, and more) to TLS. Besides, LogCollector creates index automatically during this process.

The reserved fields are as follows.

Reserved field	Description
__image_name__	Image name.
__container_name__	Container name.
__container_ip__	IP address of the container or Pod.
__pod_name__	Pod name.
__pod_uid__	Unique identifier of the Pod.
__namespace__	The Namespace to which the Pod belongs.

Prerequisites

LogCollector is installed in containers to be collected. For detailed steps, see Install LogCollector (DaemonSet deployment).

A host group is created and the host running the containers is added to the group.

We recommend that you use host group with host identification. The host identification can be set as the custom label you configured during LogCollector installation. That is to say, the identification must match the value of ${your_labels} in the ConfigMap, for example, nginx-log. For detailed steps, see Create host group (Host identification).

Limits

Sidecar mode does not support collecting stdout or stderr from application containers.

To collect text logs from application containers via Sidecar mode, you need to collect them from the shared log directory between the LogCollector and application containers. Note:

Application containers write logs to the shared directory, and LogCollector periodically scans log files in the shared directory to collect logs. Therefore, the collection path configured in the console must be the log file path from the LogCollector container.

We recommend using emptyDir to mount the collection path. For mounting details, refer to the LogCollector deployment via Sidecar.

Symbolic link collection is supported, provided that the LogCollector container can access the log files in the business container pointed to by the symbolic link.

After the Pod that contains the LogCollector and business containers stops, the LogCollector container stops running and log collection for the business container ends. If delay exists when LogCollector collects logs from the business container, some logs generated before the container stops may not be collected.

Obtaining container metadata depends on environment variables configured in the LogCollector container. LogCollector first reads the LOG_COLLECTOR_ENV_TAGS environment variables, and then splits them into multiple environment variable keys using vertical bar (|). After that, LogCollector reads the values corresponding to those keys, and then includes these key-value pairs in the logs. Therefore, you need to configure the required container metadata in the LogCollector container. However, you need to note that except for __namespace__, __node_ip__, __node_name__, __pod_ip__, and __pod_name__, other metadata must not start or end with double underscores (__); otherwise they will be ignored during collection.

Create LogCollector collection configuration

Step 1 Select a log space

Select the region where torch log service is located from the top navigation bar.

On the left navigation bar, select Common functions > Log Collection.

Then on the Log Collection page, in the LogCollector tab, click Create collection configuration.

Notice

For the regions below, manually select the log collection mode in the LogCollector tab.

Region: Hong Kong (China), Asia Pacific Southeast (Johor), Asia Pacific Southeast (Jakarta).

Select project and topic, and then click Next: Select host group.

The collected log data will be stored in the selected log topic.

Step 2 Select host group

In the All host groups section, select the specified host group for log collection.

If no host group is available in the list, you need to create a host group first. For specific instructions, see Create a machine group (machine identity), Create a Machine Group (Internet Protocol Address).

After confirming the host group in the Selected host group section, click Next Step: Collection rule.

Step 3 Configure collection rule

The collection mode determines how LogCollector parses log files. Select the mode based on your business scenario and configure the corresponding log parsing rules for the chosen mode. When collecting container logs in Sidecar mode, the supported log collection modes are as follows. You can refer to the corresponding documentation to complete the log collection.

Single-line full text mode

JSON mode

Separator mode

Multi-line full text mode

Single-line complete regex mode

Multi-line complete regex mode

NGINX configuration mode

Notice

The container log collection option is not required in Sidecar mode.

Step 4 Check index configuration

Check index configuration.

Follow the prompts on the page to set or update the index as needed. You can also click Import existing index configuration to import indexes from other log topics.

If this log topic does not have an index configured, refer to Configuration Index for setup.

If the index for this log topic is already configured, you can decide whether to update the index based on the log fields parsed by the collection rule.

Notice

You must enable the index to query and analyze the collected log data.

The updated index settings only apply to new data written afterward. For more instructions and limitations on query and analysis, see Query overview and Analysis Overview.

Click Submit.

Next steps

After creating the collection configuration, LogCollector starts monitoring log files and collecting logs based on the specified rules. TLS stores log data in the specified log topic. You can query and analyze within the log topic.

Last updated: 2025.12.05 15:12:19