Parameters | Type | Example value | Description |
HostCount | Integer | 2 | Number of machines. |
RuleCount | Integer | 1 | Number of bound collection configurations. |
AutoUpdate | Boolean | true | LogCollector installed on the host group server has auto-upgrade function. You can choose whether to enable it.
|
CreateTime | String | 2021-08-18 13:32:23 | The creation time of the host group. |
ModifyTime | String | 2021-08-18 13:32:23 | Time of host group modification. |
HostGroupId | String | 0fd********** | ID of the host group. |
HostGroupName | String | testname | Name of the host group. |
HostGroupType | String | IP | The type of host group.
|
UpdateEndTime | String | "02:00" | End time of LogCollector automatic upgrade. |
HostIdentifier | String | label | Machine identifier. |
IamProjectName | String | default | The IAM project to which the host group belongs. |
ServiceLogging | Boolean | true | You can choose whether to enable the LogCollector service log.
|
UpdateStartTime | String | "00:00" | The start time for LogCollector automatic upgrade. tip
|
AgentLatestVersion | String | "2.3.4" | The latest version number of LogCollector released by TLS. |
NormalHeartbeatStatusCount | Integer | 1 | Number of machines with normal heartbeat. |
AbnormalHeartbeatStatusCount | Integer | 1 | Number of machines with abnormal heartbeat. |
Parameters | Type | Example value | Description |
Keys | Array of String | [ "time", "", "level", "msg" ] | Log field name (Key) list.
|
Quote | String | " | Quotation marks. Content wrapped by quotation marks will not be separated, but will be parsed as a complete field. Valid only when LogType is set to delimiter_log. |
TimeKey | String | request_time | The field name for the log time field If you use a specific time field in the log as the timestamp, you must fill in both TimeKey and TimeFormat TimeKey and TimeFormat must be used together |
LogRegex | String | [(\d+-\d+-\w+:\d+:\d+,\d+)]\s[(\w+)]\s(.*) | The regular expression that the entire log entry must match.
|
TimeZone | String | GMT+08:00 | Time zone, supports host time zone (default) and custom time zone. Custom time zone supports both GMT and UTC.
|
Delimiter | String | # | Log separator. Valid only when LogType is delimiter_log. |
BeginRegex | String | [(\d+-\d+-\w+:\d+:\d+,\d+)]\s[(\w+)]\s(.*) | The regular expression to match the first line of the log.
|
TimeFormat | String | %Y-%m-%dT%H:%M:%S,%f | Time field parsing format. If you use a specified time field in the log as the log timestamp, you must provide both TimeKey and TimeFormat.
|
LogTemplate | { "Type": "Nginx", "Format": "format main '$remote_addr - $remote_user [$time_local] \"$request\" $request_time $request_length $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\"';" } | Automatically extract log fields using the specified log template. | |
UnMatchLogKey | String | LogParseFailed | When uploading logs that failed to be parsed, this is the key name for logs that failed to be parsed. UnMatchUpLoadSwitch=true and UnMatchLogKey must be used together. |
FilterKeyRegex | / | Filter rule list.
| |
TimeExtractRegex | String | [0-9]{0,2}\/[0-9a-zA-Z]+\/[0-9:,]+ | Regular expression for extracting time, used to extract the time value from the TimeKey field and parse it as the corresponding collection time. |
UnMatchUpLoadSwitch | Boolean | true | Whether to upload logs that failed to be parsed. UnMatchUpLoadSwitch=true and UnMatchLogKey must be used together.
|
EnableNanosecond | Boolean | true | Enable nanosecond precision time. When enabled, log time will be parsed and reported with nanosecond precision.
|
TimeSample | String | 2006-01-02T15:04:05Z07:00 | Time sample. Used to check whether the provided time parsing format is correct. |
Parameters | Type | Example value | Description |
EnvTag | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | This feature determines whether to set environment label as log tag and add it to raw log data. If enabled, TLS will add related container environment variable fields to the logs. If you set multiple key-value pairs, it means you add multiple fields. For example, set Key as source and Value as data_source. If the container includes the environment variable source=DC, a new field __tag__data_source__: DC will be added to the log. |
Stream | String | all | Collection details.
|
KubernetesRule | { "NamespaceNameRegex": ".*test.*", "WorkloadType": "Deployment", "WorkloadNameRegex": ".*test.*", "IncludePodLabelRegex": { "Key1": "Value1", "Key2": "Value2" }, "ExcludePodLabelRegex": { "Key1": "Value1", "Key2": "Value2" }, "PodNameRegex": ".*test.*", "LabelTag": { "Key1": "Value1", "Key2": "Value2" } } | Kubernetes container collection rules. | |
ContainerNameRegex | String | .*test.* | Name of container to collect If the container name is not specified, all containers in the host group will be collected. Supports regex matching. For example, set the container name to ^(container-test)$ to collect all containers named container-test. |
ExcludeContainerEnvRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | The container environment variable denylist specifies which containers are excluded from data collection. If the denylist is disabled, all containers will be collected. If you enable the container environment variable denylist, key is required, while value is optional.
|
IncludeContainerEnvRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | Specify which containers to collect by using the container environment variable allowlist. If the allowlist is not enabled, all containers will be collected. To enable the container environment variable allowlist, enter key-value pairs. Key is required; value is optional.
|
ExcludeContainerLabelRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | The container label denylist is used to specify which containers should not be collected. If the denylist is disabled, all containers are collected. If the container label denylist is enabled, the key field is required and the value field is optional.
|
IncludeContainerLabelRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | You can specify which containers to collect using container label allowlists. When the allowlist is not enabled, all containers will be collected. When enabling the container label allowlist, you need to fill in key-value pairs. Key is required; value is optional.
|
Parameters | Type | Example value | Description |
Fields | JSON Map | {"ClusterID":"dabaad5f-7a10-4771-b3ea-d821f73e****"} | Add constant fields to logs. Constant fields must follow these restrictions:
|
Plugin | {"processors":[{"json":{"field":"__content__","trim_keys":{"mode":"all","chars":"#"},"trim_values":{"mode":"all","chars":"#"},"allow_overwrite_keys":true,"allow_empty_values":true} | Plugin configuration for LogCollector After enabling plugin configuration, you can add one or more LogCollector processor plugins to parse logs with complex or irregular structures. | |
Advanced | { "CloseInactive": 10 } | LogCollector extended configuration. | |
RawLogKey | String | raw | Original log field name. Effective only when EnableRawLog is set to true. RawLogKey defaults to __raw__, meaning the original log data will be encapsulated in the __raw__ field and uploaded to TLS along with the parsed log data. |
TailFiles | Boolean | true | LogCollector collection policy specifies whether LogCollector collects incremental logs or full logs. The default value is false, meaning full log collection.
|
HostnameKey | String | hostname | hostname field name. Only required when EnableHostname is set to true. |
EnableRawLog | Boolean | true | Upload original log.
|
ShardHashKey | { "HashKey": "3C" } | Rules for routing log shards.
| |
ParsePathRule | { "PathSample": "/data/nginx/log/dabaad5f-7a10/tls/app.log", "Regex": "\\/data\\/nginx\\/log\\/(\\w+)-(\\w+)\\/tls\\/app\\.log", "Keys": ["instance-id", "pod-name"] } | Parse collection path rules. After setting the rules, fields will be extracted from the collection path using the specified regular expressions in the rules and added as metadata to the log data. tip This parameter is not supported when collecting container standard output. | |
EnableHostname | Boolean | true | This feature determines whether to upload the hostname field. By default, it is disabled.
|
HostGroupLabelKey | String | host_group_label | The field name used to store host group label information. |
EnableHostGroupLabel | Boolean | true | You can select whether to upload the label information of the host group to TLS. By default, it is enabled.
|
TailSizeKb | Integer | 1024 | Backtracking threshold for incremental collection, measured in KiB. When the LogCollector collection strategy is incremental, for the first collection of a log file:
|
IgnoreOlder | Integer | 7 | Ignore log files not updated for a certain period; the unit is hours. |
MultiCollectsType | String | RuleID | Collect log files multiple times.
|
Parameters | Type | Example value | Description |
LabelTag | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | This feature determines whether to add Kubernetes Labels as log tags to the raw log data. If enabled, TLS will add Kubernetes Pod Label related fields to the logs. If you set multiple key-value pairs, multiple fields are added. For example, set the key to source and the value to data_source. If the Pod includes the label source=DC, the log will have a new field __tag__data_source__: DC. |
PodNameRegex | String | .*test.* | Pod name is used to specify the target container for collection. If you do not specify the Pod name, all containers are collected. Pod name supports regex matching. For example, if you set the Pod name to ^(http.*)$, it collects all containers under Pods whose names start with "http". |
WorkloadType | String | Deployment | Specify the container for collection based on workload type. Only one type can be selected. If no type is specified, containers of all types are collected. Supported workload types:
|
AnnotationTag | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | This feature determines whether to set Kubernetes Annotation to log tags and add it into the raw log data. If enabled, TLS will add Kubernetes Pod annotation fields to the logs. If you set multiple key-value pairs, multiple fields are added. If you set the key to "sink" and the value to "data_sink", and the Pod has an annotation sink=ck, a new field __tag__data_sink__: ck will be added to the log. |
WorkloadNameRegex | String | .*test.* | By configuring the workload name, you can specify the container to collect. If no workload name is specified, all containers are collected. Workload names support regular expression matching. For example, if you set the workload name to ^(http.*)$, all containers under workloads starting with http will be collected. |
NamespaceNameRegex | String | .*test.* | Name of the Kubernetes namespace to be collected. If no Namespace is specified, all containers will be collected. Namespace names support regular expression matching. For example, setting the Namespace name to ^(tcp|udp)$ collects all containers under the tcp and udp namespaces. |
ExcludePodLabelRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | The Pod Label denylist is used to specify which containers should not be collected. If the denylist is disabled, all containers are collected. If you need to set a Pod Label denylist, the key is required, and the value is optional.
tip
|
IncludePodLabelRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | The Pod Label allowlist is used to specify the containers to be collected. If the Pod Label allowlist is not enabled, all containers will be collected. If you need to set the Pod label allowlist, the key field is required.
tip
|
EnableAllLabelTag | Boolean | true | This feature determines whether to add all Kubernetes labels as log tags to the raw log data. When enabled, TLS will add all related Kubernetes Pod label fields to logs. For example, if a Pod contains the label source=DC and destination=CS, new fields __tag__source__: DC and __tag__destination__: CS will be added to the log. |
ExcludePodAnnotationRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | Pod Annotation denylist specifies containers that are excluded from collection. If you need to set a Pod Annotation denylist, the key is required and the value is optional.
tip
|
IncludePodAnnotationRegex | JSON Map | { "Key1": "Value1", "Key2": "Value2" } | Pod annotation allowlist is used to specify the containers to be collected. If you need to set a Pod annotation allowlist, the key field is required and the value field is optional.
tip
|
Parameters | Type | Example value | Description |
CloseEOF | Boolean | true | After reading to the end of the log file, release the handle for that log file. The default value is false. |
CloseRemoved | Boolean | true | Release the handle of the log file after it is removed. The default value is false. |
CloseRenamed | Boolean | true | After the log file is renamed, release the handle for that log file. The default value is false. |
CloseTimeout | Integer | 1800 | Maximum monitoring duration for LogCollector log files. Measured in seconds. The default value is 0 seconds, which means LogCollector does not restrict the monitoring duration of log files. When LogCollector starts monitoring a log file, timing begins. If the specified duration is exceeded, LogCollector immediately releases the log file handle and stops monitoring, whether the log file has been fully read or not. |
CloseInactive | Integer | 10 | Wait time before releasing log file handle. If no new log entries are written to the log file within the specified time, the log file handle will be released. The unit is seconds. The valid range is 1–300 seconds, with a default value of 60 seconds. |
NoLineTerminatorEOFMaxTime | Integer | 5 | Maximum wait time when LogCollector does not read a newline character. Unit: seconds. Default is 5s. LogCollector starts timing when it begins reading the log file. If no newline character is found within the specified time, it sends the logs in the buffer. If the interval between file writes is long, a complete log entry may be split into two parts when written to disk. Adjust this parameter according to your log write interval. |