Docs

Torch Log Service

Installation and upgrade

Install LogCollector (DaemonSet deployment)

Before collecting container text logs or standard output, you need to install LogCollector in the Kubernetes cluster and configure the relevant environment. This topic introduces how to install LogCollector using DaemonSet deployment.

Background information

After installing LogCollector on a Kubernetes cluster via DaemonSet deployment, LogCollector can collect text logs or standard output from containers inside other Pods. The following steps are mainly required during installation.

Configure ServiceAccount, ClusterRole, and ClusterRoleBinding; complete LogCollector authentication and authorization.

Create ConfigMap for LogCollector.

Create LogCollector DaemonSet.

Preparation

The IAM user has been created, and the Access Key ID and Secret Access Key have been obtained. Besides, the user is granted at least the TLSLogCollectorAccess permission policy. For details about permissions, see IAM overview.

How to install Helm

Notice

Helm 3 client is installed (version 3.7 or later version). For detailed instructions, see Helm.

1 Download LogCollector Chart package

Log in to the Kubernetes cluster where you need to install LogCollector. Run the corresponding download command based on the cluster region to download the LogCollector Chart package.

Notice

Region ID: Replace this with the actual region ID for your environment. For example, cn-beijing for North China 2 (Beijing).

--version 0.1.0: Used to obtain parameters for the LogCollector chart; this value is fixed.

LogCollector version: The version is determined by the image parameter specified in the LogCollector Chart deployment command. By default, the latest LogCollector version is used.

2 Deploy the LogCollector Chart package

Run the following command to deploy the LogCollector Chart package.

Notice

If prompt shows Error: INSTALLATION FAILED: Unable to continue with install: ClusterRole "logcollector" in namespace "" exists and can not be imported into the current release…, a ClusterRole with the same name has already been installed. You need to run kubectl delete clusterrole logcollector to delete this ClusterRole first.

If prompt shows Error: INSTALLATION FAILED: Unable to continue with install: ClusterRoleBinding "logcollector" in namespace "" exists and cannot be imported into the current release…, it means that a ClusterRoleBinding with the same name has already been installed. You need to run kubectl delete clusterrolebinding logcollector to delete this ClusterRoleBinding first.

The following are required parameters: region, endpoint, secret_id, secret_key, label, image, logcollector_data. Details are provided below.

Parameters	Required	Example value	Description
region	Yes	cn-beijing	Region ID where the project is located. For more details, see Service address.
endpoint	Yes	https://tls-cn-beijing.volces.com	TLS connection domain, i.e., the service address. Log in to Torch Log Service console, and you can view the connection domain name on the project details page.
secret_id	Yes	AK********	Access Key ID for the Volcengine account or IAM user. You can obtain the Access Key ID on Access Key Management on IAM.
secret_key	Yes	TUdZ********	Secret Access Key for the VolcanoEngine account or IAM user. You can obtain the Secret Access Key on Key management page on IAM.
label	Yes	http_module	Label identifier for LogCollector. If you need to add this LogCollector to a host group of host identification type, the label parameter must be set. You can configure multiple label identifiers; separate them with commas (,).
namespace	Yes	kube-system	LogCollector namespace. Only one can be specified; and the namespace must already exist. If the namespace does not exist, deployment will fail.
image	Yes	logcollector-cn-beijing.cr.volces.com/logcollector/logcollector:latest	LogCollector image address.
logcollector_data	Yes	/var/lib/logcollector_data	Specifies the node directory for storing LogCollector CheckPoint status. This ensures LogCollector does not collect duplicate logs after Pod restarts.

Shell script installation method

1 Download the LogCollector installation package

Log in to the Kubernetes cluster where you need to install LogCollector; download the LogCollector installation package to the specified directory according to your cluster’s region and network type.

Replace Region ID in the download address with the ID for your actual region.

Private network download address

wget https://logcollector-<Region ID>.tos-<Region ID>.ivolces.com/logcollector.tgz; tar -xzf logcollector.tgz; chmod 755 ./logcollector/logcollector.sh

Public network download address

wget https://logcollector-<Region ID>.tos-<Region ID>.volces.com/logcollector.tgz; tar -xzf logcollector.tgz; chmod 755 ./logcollector/logcollector.sh

2 Install LogCollector

Run the following command to install LogCollector.

Among the parameters, region, endpoint, secret_id, secret_key, label, and logcollector_data are required parameters; namespace and image are optional parameters. Details are as follows.

Parameters	Required	Example value	Description
region	Yes	cn-beijing	Region ID where the project is located. For more details, see Service address.
endpoint	Yes	https://tls-cn-beijing.volces.com	TLS connection domain, i.e., the service address. Log in to Torch Log Service console, and you can view the connection domain name on the project details page. For more details, see Service address.
secret_id	Yes	AK********	Access Key ID for the Volcengine account or IAM user. You can obtain the Access Key ID on Access Key Management on IAM.
secret_key	Yes	TUdZ********	Secret Access Key for the VolcanoEngine account or IAM user. You can obtain the Secret Access Key on Key management page on IAM.
label	Yes	http_module	Label identifier for LogCollector. If you need to add this LogCollector to a host group of host identification type, the label parameter must be set. When creating a host group of Host identification type, set Host identification as the label. For details, see Host group overview. You can configure multiple label identifications; separate them by commas (,).
namespace	No	kube-system	LogCollector namespace. If a namespace is not specified, the kube-system namespace will be used. If no namespace exists, a new namespace will be created.
image	No	logcollector-cn-beijing.cr.volces.com/logcollector/logcollector:latest	LogCollector image address. If no image address is specified, the latest image address for the selected region will be used. For the latest image address, see LogCollector image address.
logcollector_data	Yes	/var/lib/logcollector_data	Specifies the node directory for storing LogCollector CheckPoint status. This ensures LogCollector does not collect duplicate logs after Pod restarts.

Manual installation method

1 Create Namespace

This step creates a Namespace .

Create a YAML file named logcollector_ns.yaml .

vim logcollector_ns.yaml

Edit the YAML file and enter the following content.

Configure the following parameters as needed.

apiVersion: v1
kind: Namespace
metadata:
name: <namespace>

The parameters are described below:

Parameters
Example value
Description
name
kube-system
Namespace name.

Run the following command to apply the Namespace configuration.

kubectl apply -f logcollector_ns.yaml

2 Create ServiceAccount

Create a ServiceAccount for LogCollector and configure basic identity information.

Create a YAML file named logcollector_sa.yaml.

vim logcollector_sa.yaml

Enter the following content in the YAML file.

apiVersion: v1
kind: ServiceAccount
metadata:
name: logcollector
namespace: <namespace>

Parameters
Example value
Description
namespace
kube-system
Namespace. You can only specify one namespace, and it must match the namespace set in step 1.

Run the following command to apply the ServiceAccount configuration.

kubectl apply -f logcollector_sa.yaml

3 Create ClusterRole

Create a ClusterRole for LogCollector.

Create a YAML file named logcollector_cr.yaml.

vim logcollector_cr.yaml

Enter the following content in the YAML file.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: logcollector
rules:
- apiGroups:
- ""
resources:
- pods
- nodes
- namespaces
verbs:
- get
- list
- watch
- apiGroups:
- apps
resources:
- replicasets
- deployments
- daemonsets
- statefulsets
verbs:
- watch
- list
- get
- apiGroups:
- batch
resources:
- jobs
- cronjobs
verbs:
- watch
- list
- get

Run the following command to apply the ClusterRole configuration.

kubectl apply -f logcollector_cr.yaml

4 Create ClusterRoleBinding

Create a ClusterRoleBinding for LogCollector.

Create a YAML file named logcollector_crb.yaml.

vim logcollector_crb.yaml

Enter the following content in the YAML file.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: logcollector
subjects:
- kind: ServiceAccount
name: logcollector
namespace: <namespace>
roleRef:
kind: ClusterRole
name: logcollector
apiGroup: rbac.authorization.k8s.io

The parameters are described below:

Parameters
Example value
Description
namespace
kube-system
Namespace. You can only specify one, and it must match the namespace set in Step 1.

Run the following command to apply the ClusterRoleBinding configuration.

kubectl apply -f logcollector_crb.yaml

5 Create ConfigMap

Create a ConfigMap for LogCollector.

Create a YAML file named logcollector_config.yaml.

vim logcollector_config.yaml

Enter the following content in the YAML file.

apiVersion: v1
kind: ConfigMap
metadata:
name: logcollector-config
namespace: <namespace>
data:
logcollector.yml: |
daemonset_mode: true

autodiscover:
providers:
- type: kubernetes
node: ${NODE_NAME}

output:
tls:
region: <region>
endpoint: <endpoint>
secret_id: <secret_id>
secret_key: <secret_key>
label: <label>

The parameters are described below:

Parameters	Example value	Description
namespace	kube-system	Namespace: Enter the namespace you set in 1 Create Namespace.
region	cn-beijing	Region ID where the project is located. For more details, see Service address.
endpoint	https://tls-cn-beijing.volces.com	TLS connection domain, i.e., the service address. Log in to Torch Log Service console, and you can view the connection domain name on the project details page. For more details, see Service address.
secret_id	AK********	Access Key ID for the Volcengine account or IAM user. You can obtain the Access Key ID on Access Key Management on IAM.
secret_key	TUdZ********	Secret Access Key for the VolcanoEngine account or IAM user. You can obtain the Secret Access Key on Key management page on IAM.
label	http_module	Label identifier for LogCollector. If you need to add this LogCollector to a host group of host identification type, you must set the label parameter. When creating a host group of host identification type, set Host identification as the label. For details, see Host group overview. You can configure multiple label identifications; separate them by commas (,).

Run the following command to apply the ConfigMap configuration.

kubectl apply -f logcollector_config.yaml

6 Create DaemonSet

Create a DaemonSet for LogCollector to collect logs from the current broker.

Create a YAML file named logcollector_ds.yaml.

vim logcollector_ds.yaml

Enter the following content in the YAML file and replace the parameters as needed.

apiVersion: apps/v1
kind: DaemonSet
metadata:
name: logcollector
namespace: <namespace>
labels:
k8s-app: logcollector
app.kubernetes.io/instance: logcollector
spec:
selector:
matchLabels:
k8s-app: logcollector
template:
metadata:
labels:
k8s-app: logcollector
spec:
serviceAccountName: logcollector
terminationGracePeriodSeconds: 65
containers:
- name: logcollector
image: <image>
imagePullPolicy: Always
args:
- /usr/local/container/logcollector/logcollectord
- -c
- /usr/local/container/logcollector/etc/logcollector.yml
- --httpprof
- 0.0.0.0:6060
securityContext:
runAsUser: 0
# If using Red Hat OpenShift, uncomment this:
# privileged: true
resources:
limits:
cpu: '2'
memory: 2Gi
requests:
cpu: '1'
memory: 1Gi
volumeMounts:
- name: logcollector-config
mountPath: /usr/local/container/logcollector/etc
readOnly: true
- name: logcollector-host
mountPath: /logcollector_host
mountPropagation: HostToContainer
readOnly: true
- name: logcollector-data
mountPath: /usr/local/container/logcollector/data
env:
- name: MAX_MEM
valueFrom:
resourceFieldRef:
containerName: logcollector
resource: limits.memory
- name: MAX_PROCS
valueFrom:
resourceFieldRef:
containerName: logcollector
resource: limits.cpu
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: NODE_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
volumes:
- name: logcollector-config
configMap:
defaultMode: 420
name: logcollector-config
- name: logcollector-host
hostPath:
path: /
- name: logcollector-data
hostPath:
path: <logcollector_data>
type: DirectoryOrCreate

The parameters are described below:

Parameters	Example value	Description
namespace	kube-system	Namespace. Enter the namespace you set in 1 Create Namespace.
image	logcollector-cn-beijing.cr.volces.com/logcollector/logcollector:latest	LogCollector image address For the latest version image address, refer to LogCollector image address.
path	/var/lib/logcollector_data	Stores the node directory for LogCollector CheckPoint status to ensure that LogCollector does not duplicate log collection after a Pod restart.

Run the following command to apply the DaemonSet configuration.

kubectl apply -f logcollector_ds.yaml

LogCollector image address

The latest LogCollector image addresses provided by Torch log service (TLS) in each region are as follows:

Replace Region ID in the image address with the actual region ID for your environment, such as cn-beijing for North China 2 (Beijing).

Last updated: 2026.09.11 11:34:05