Docs

Torch Log Service

Collect container logs

Collect Docker container logs

TLS supports collecting Docker container logs. This topic describes how to collect Docker container logs by creating a host group with Host identification type.

Background information

After you install the LogCollector plugin provided by TLS in the container and create a collection configuration, LogCollector will do the following: identifies the containers to collect based on container labels and environment variables, continuously collects text logs and standard output generated in the specified container path, and uploads log data along with container names and other container-related metadata to the server. In addition, LogCollector also supports breakpoint resume. The CheckPoint file retains collection location information. Therefore, even if LogCollector is stopped and then restarted, it continues collecting from the last saved location.

Prerequisites

The IAM role is created, and the Access Key ID and Secret Access Key are obtained. The user is granted TLSFullAccess permissions for the log topic. For more details, see IAM overview.

Limits

Items	Description
Version limits	LogCollector: Supports V2.3.6 and later versions. Docker API version: Supports versions 1.40 to 1.51.
Standard output collection limits	The parameter "log-driver":"json-file" is required to add to the Docker configuration file daemon.json. If you are using CentOS 7.4 or later, you need to set fs.may_detach_mounts=1.
Text log collection limits	Supports Docker storage drivers overlay and overlay2. Other types of storage drivers require manually mounting the log directory.

Steps

1 Install LogCollector

Obtain the LogCollector image.

Run the command below on a host with Docker installed to fetch the LogCollector image.

docker pull logcollector-Region ID.cr.volces.com/logcollector/logcollector:latest

Description

Replace Region ID in the image address with the actual region ID for your environment. For example, the Region ID for North China 2 (Beijing) is cn-beijing. For details, see Service address.

Create a LogCollector configuration file to set up log service connection details.

Create a local directory for the LogCollector configuration files, such as /usr/local/container/logcollector/etc.

mkdir -p /usr/local/container/logcollector/etc

Create the configuration file logcollector.yaml.

cd /usr/local/container/logcollector/etc
touch logcollector.yaml

The configuration file should contain the following content:

docker_mode: true
autodiscover:
providers:
- type: docker
cleanup_timeout: 15s

output:
tls:
region: <region>
endpoint: <endpoint>
secret_id: <secret_id>
secret_key: <secret_key>
label: <label>

The tls parameter is explained as follows:

Parameters	Required	Example value	Description
region	Yes	cn-beijing	The region ID where the project is located.
endpoint	Yes	https://tls-cn-beijing.volces.com	TLS connection domain, i.e., the service address. After logging into TLS Console, you can view the connection domain name on the project details page. For details, see Service address.
secret_id	Yes	AK********	Access Key ID for the Volcengine account or IAM user. You can obtain the Access Key ID on the Key management page in the Volcengine Console.
secret_key	Yes	TUdZ********	Secret Access Key for the VolcanoEngine account or IAM user. You can obtain the Secret Access Key on Key management page on IAM.
label	Yes	http_module	Host identification of the LogCollector. This is used to add the LogCollector to a host group of Host identification type. You can configure multiple label identifiers; separate the identifiers by commas (,).

Start the container and make sure the directories are mounted correctly.

docker run -d\
--name logcollector \
-u 0 \
-v /:/logcollector_host:ro \
-v /var/run/docker.sock:/var/run/docker.sock \
-v {logcollector-data-dir}:/usr/local/container/logcollector/data \
-v {logcollector-config-dir}:/usr/local/container/logcollector/etc \
{image} \
/usr/local/container/logcollector/logcollectord \
-c \
/usr/local/container/logcollector/etc/logcollector.yaml

Description of startup command parameters:

Parameters	Required	Example value	Description
logcollector-data-dir	Yes	/var/lib/logcollector/docker-data	This directory stores LogCollector checkpoint data to ensure that LogCollector does not collect logs repeatedly after a container restarts.
logcollector-config-dir	Yes	/usr/local/container/logcollector/etc	This is the directory where the configuration file logcollector.yaml is created.
image	Yes	logcollector-cn-beijing.cr.volces.com/logcollector/logcollector:latest	LogCollector image address.

Verify the LogCollector startup status.

Run the following command to verify the LogCollector startup status:

docker ps | grep logcollector

If the output matches the following, LogCollector has started successfully:

96fb3bc150f9 logcollector-cn-beijing.cr.volces.com/logcollector/logcollector:v2.3.6 "/usr/local/containe…" About a minute ago Up About a minute logcollector

2 Create a host group

In TLS console, create a Host identification type host group.

Set some parameters of the host group as follows. Configure other parameters based on your actual environment. For details, you can refer to Create host group (Host identification) to create a host group.

Parameters	Example value	Description
Host group type	Host identification	Select the type of host group. Select Host identification.
Host identification	http_module	Enter the host identification of the LogCollector configured in 1 Install LogCollector.

Verify the host group status.

On the Host group management tab, click the name of the newly created host group to enter the details page. Check the heartbeat status on the Heartbeat status tab:

If the heartbeat is normal and the IP count meets expectations, it means that LogCollector has successfully connected to TLS.

If any other exception exists, you can check whether the LogCollector configuration file is set correctly.

3 Create log collection configuration

In TLS console, create a project, topic, and log collection configuration. The specific approach depends on the log collection type as follows:

Collection type	Description	Create log collection configuration
Container application logs	Application logs within the container are output to standard output (stdout) and standard error (stderr).	See Collect container standard output (DaemonSet - Console method).
Container text logs	Logs written by applications in containers, Docker engine logs (daemon), Docker event logs (events), and container metadata logs.	Refer to Collect container text logs (DaemonSet - Console method).

Description of some parameters in the collection rules:

Basic information	Description
Container log collection	When collecting Docker container logs, select Enable.
Collection type	Set the collection type for container logs and choose the type based on actual needs. K8s container standard output: Collects application logs from containers. K8s container text logs: Collects text logs from containers.
Universal rules for container collection	Specify the content to collect based on general container information, or exclude certain content from collection at the same time. If this parameter is not enabled, it means that all containers are included in the collection scope.
K8s collection rule	This parameter is not required to configured when collecting Docker container logs.

Last updated: 2026.06.02 17:45:18