Parameters | Type | Example value | Description |
Keys | Array of String | [ "time", "", "level", "msg" ] | List of log field names (Key).
|
Quote | String | " | Quotation marks. Content wrapped by quotation marks will not be separated but will be parsed as a complete field. Valid only when LogType is delimiter_log. |
TimeKey | String | request_time | Name of the log time field. If you use a specific time field in the log as the log timestamp, you must set both TimeKey and TimeFormat. TimeKey and TimeFormat must be provided together. |
LogRegex | String | [(\d+-\d+-\w+:\d+:\d+,\d+)]\s[(\w+)]\s(.*) | The regular expression required for the entire log.
|
TimeZone | String | GMT+08:00 | Time zone: supports Host time zone (default) and Custom time zone Custom time zone supports both GMT and UTC
|
Delimiter | String | # | Log separator. Applies only when LogType is set to delimiter_log. |
BeginRegex | String | [(\d+-\d+-\w+:\d+:\d+,\d+)]\s[(\w+)]\s(.*) | Regular expression required to match the first line of the log
|
TimeFormat | String | %Y-%m-%dT%H:%M:%S,%f | Time field parsing format. If you want to use a specific time field in the log as the log timestamp, you need to specify both TimeKey and TimeFormat.
|
LogTemplate | { "Type": "Nginx", "Format": "format main '$remote_addr - $remote_user [$time_local] \"$request\" $request_time $request_length $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\"';" } | Automatically extract log fields based on the specified log template. | |
UnMatchLogKey | String | LogParseFailed | When uploading logs that fail to parse, the key name for the failed logs. UnMatchUpLoadSwitch=true and UnMatchLogKey must be used together. |
FilterKeyRegex | / | Filter rule list.
| |
TimeExtractRegex | String | [0-9]{0,2}\/[0-9a-zA-Z]+\/[0-9:,]+ | Regular expression for extracting time, used to extract the time value from the TimeKey field and parse it as the corresponding collection time. |
UnMatchUpLoadSwitch | Boolean | true | Upload logs that failed to parse. UnMatchUpLoadSwitch=true and UnMatchLogKey must appear together.
|
EnableNanosecond | Boolean | true | Enable nanosecond precision time function. When enabled, log time is parsed and reported with nanosecond precision.
|
TimeSample | String | 2006-01-02T15:04:05Z07:00 | Time example. Checks if the entered time parsing format is correct. |
Parameters | Type | Example value | Description |
EnvTag | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | This feature determines whether to set environment label as log tag and add it to raw log data. If enabled, TLS will add related container environment variable fields to the logs. If you set multiple key-value pairs, it means you add multiple fields. For example, set the key to source and the value to data_source. If the container includes the environment variable source=DC, a new field __tag__data_source__: DC will be added to the log. |
Stream | String | all | Collection details
|
KubernetesRule | { "NamespaceNameRegex": ".*test.*", "WorkloadType": "Deployment", "WorkloadNameRegex": ".*test.*", "IncludePodLabelRegex": { "Key1": "Value1", "Key2": "Value2" }, "ExcludePodLabelRegex": { "Key1": "Value1", "Key2": "Value2" }, "PodNameRegex": ".*test.*", "LabelTag": { "Key1": "Value1", "Key2": "Value2" } } | Kubernetes container collection rule. | |
ContainerNameRegex | String | .*test.* | Name of container to collect If the container name is not specified, all containers in the host group will be collected. Supports regular expressions. For example, set the container name to ^(container-test)$ to collect all containers named container-test. |
ExcludeContainerEnvRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | The container environment variable denylist specifies which containers are excluded from data collection. If the denylist is disabled, all containers will be collected. If you enable the container environment variable denylist, key is required, while value is optional.
|
IncludeContainerEnvRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | Specify which containers to collect by using the container environment variable allowlist. If the allowlist is not enabled, all containers will be collected. To enable the container environment variable allowlist, enter key-value pairs. Key is required; value is optional.
|
ExcludeContainerLabelRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | The container label denylist is used to specify which containers should not be collected. If the denylist is disabled, all containers are collected. If the container label denylist is enabled, the key field is required and the value field is optional.
|
IncludeContainerLabelRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | You can specify which containers to collect using container label allowlists. When the allowlist is not enabled, all containers will be collected. When enabling the container label allowlist, you need to fill in key-value pairs. Key is required; value is optional.
|
Parameters | Type | Example value | Description |
Fields | JSON Map | {"ClusterID":"dabaad5f-7a10-4771-b3ea-d821f73e****"} | Add constant fields to logs Constant fields must follow these restrictions:
|
Plugin | {"processors":[{"json":{"field":"__content__","trim_keys":{"mode":"all","chars":"#"},"trim_values":{"mode":"all","chars":"#"},"allow_overwrite_keys":true,"allow_empty_values":true} | Plugin configuration for LogCollector. After enabling plugin configuration, you can add one or more LogCollector processor plugins to parse logs with complex or dynamic structures. | |
Advanced | { "CloseInactive": 10 } | LogCollector extended configuration. | |
RawLogKey | String | raw | Original log field name. This setting only applies when EnableRawLog is set to true. The RawLogKey defaults to __raw__, meaning the raw log data will be encapsulated in the __raw__ field and uploaded to TLS along with the parsed log data. |
TailFiles | Boolean | true | LogCollector collection policy specifies whether LogCollector collects incremental logs or full logs. The default value is false, indicating collection of all logs.
|
HostnameKey | String | hostname | hostname field name. Required only when EnableHostname is set to true. |
EnableRawLog | Boolean | true | Whether to upload the original log.
|
ShardHashKey | { "HashKey": "3C" } | Rules for routing log shard partitioning.
| |
ParsePathRule | { "PathSample": "/data/nginx/log/dabaad5f-7a10/tls/app.log", "Regex": "\\/data\\/nginx\\/log\\/(\\w+)-(\\w+)\\/tls\\/app\\.log", "Keys": ["instance-id", "pod-name"] } | Parse collection path rules. After setting the rules, extract fields from the collection path specified by the regular expression and add them as metadata to the log data. tip This parameter is not supported when collecting container standard output. | |
EnableHostname | Boolean | true | This feature determines whether to upload the hostname field. By default, it is disabled.
|
HostGroupLabelKey | String | host_group_label | Field name for storing host group Label information. |
EnableHostGroupLabel | Boolean | true | You can select whether to upload the label information of the host group to TLS. By default, it is enabled.
|
TailSizeKb | Integer | 1024 | Backtracking threshold for incremental collection, measured in KiB. When the LogCollector uses the incremental collection strategy, for the first collection of a log file:
|
IgnoreOlder | Integer | 7 | Ignore log files that have not been updated for a specified period, in hours. |
MultiCollectsType | String | RuleID | Log files can be collected multiple times.
|
Parameters | Type | Example value | Description |
LabelTag | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | This feature determines whether to add Kubernetes Labels as log tags to the raw log data. If enabled, TLS will add Kubernetes Pod Label related fields to the logs. If you set multiple key-value pairs, multiple fields are added. For example, if you set the Key as source and the Value as data_source, and the Pod contains the Label source=DC, then the log will include a new field __tag__data_source__: DC. |
PodNameRegex | String | .*test.* | Pod name is used to specify the containers to collect If no Pod name is specified, all containers will be collected Pod name supports regular expressions. For example, set the Pod name to ^(http.*)$ to collect all containers under Pods that start with "http" |
WorkloadType | String | Deployment | Specify the container for collection based on workload type. Only one type can be selected. If no type is specified, containers of all types are collected. Supported workload types:
|
AnnotationTag | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | This feature determines whether to set Kubernetes Annotation to log tags and add it into the raw log data. If enabled, TLS will add Kubernetes Pod annotation fields to the logs. If you set multiple key-value pairs, multiple fields are added. If you set the key to "sink" and the value to "data_sink", and the Pod has an annotation sink=ck, a new field __tag__data_sink__: ck will be added to the log. |
WorkloadNameRegex | String | .*test.* | By configuring the workload name, you can specify the container to collect. If no workload name is specified, all containers are collected. Workload names support regular expression matching. For example, if you set the workload name to ^(http.*)$, all containers under workloads starting with http will be collected. |
NamespaceNameRegex | String | .*test.* | Name of the Kubernetes namespace to be collected. If no Namespace is specified, all containers will be collected. Namespace names support regular expression matching. For example, setting the namespace name to ^(tcp|udp)$ means collecting all containers in the tcp namespace and udp namespace. |
ExcludePodLabelRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | The Pod Label denylist is used to specify which containers should not be collected. If the denylist is disabled, all containers are collected. If you need to set a Pod Label denylist, the key field is required, while the value field is optional.
tip
|
IncludePodLabelRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | The Pod Label allowlist is used to specify the containers to be collected. If the Pod Label allowlist is not enabled, all containers will be collected. If you need to set the Pod label allowlist, the key field is required.
tip
|
EnableAllLabelTag | Boolean | true | This feature determines whether to add all Kubernetes labels as log tags to the raw log data. When enabled, TLS will add all related Kubernetes Pod label fields to logs. For example, if the Pod contains source=DC and destination=CS labels, the log will include the fields __tag__source__: DC and __tag__destination__: CS. |
ExcludePodAnnotationRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | Pod Annotation denylist specifies containers that are excluded from collection. If you need to set a Pod Annotation denylist, the key is required and the value is optional.
By default, value uses string matching. Containers only match if the value is exactly the same as the value of the Pod annotation. If the value starts with ^ and ends with $, it uses regular expression match. For example, if you set key as app and value as ^(test1|test2)$, it matches containers within Pods whose Annotation contains app:test1 or app:test2. tip
|
IncludePodAnnotationRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | Pod annotation allowlist is used to specify the containers to be collected. If you need to set a Pod annotation allowlist, the key field is required and the value field is optional.
By default, value uses string matching. Containers only match if the value is exactly the same as the value of the Pod annotation. If the value starts with ^ and ends with $, it uses regular expression match. For example, set the key as "app" and set the value as ^(test1|test2)$, and then the containers in Pods whose annotation contains app:test1 or app:test2 will match. tip
|
Parameters | Type | Example value | Description |
CloseEOF | Boolean | true | After reading to the end of the log file, release the file handle. The default value is false. |
CloseRemoved | Boolean | true | Release the handle of the log file after it is removed. The default value is false. |
CloseRenamed | Boolean | true | After the log file is renamed, its handle will be released or not The default value is false. |
CloseTimeout | Integer | 1800 | Maximum duration LogCollector monitors log files. Unit: seconds. The default is 0 seconds, which means there is no limit on how long LogCollector monitors log files. LogCollector starts timing when it begins monitoring a log file, and releases the file handle immediately after exceeding the specified duration, regardless of whether the log file has been fully read. |
CloseInactive | Integer | 10 | Waiting time to release the log file handle. If no new logs are written to the log file after the specified time, the handle for the log file is released. The unit is seconds. Valid values range from 1 to 300 seconds, and the default is 60 seconds. |
NoLineTerminatorEOFMaxTime | Integer | 5 | Maximum wait time for LogCollector when no line break is detected. The unit is seconds, with a default value of 5s. LogCollector starts timing when it begins reading the log file. If no newline character is found within the specified duration, the buffered logs will be sent. If the interval between log file writes is large, a complete log entry may be split and written in two parts. You can adjust this parameter according to your log write interval. |