Call the DescribeHostGroupRules API to obtain the list of collection configurations already applied to the host group.

Instructions for use

This API retrieves the list of collection configurations already applied to the host group and supports paginated queries. The call rate limit is 20 requests per second; exceeding this limit will result in an ExceedQPSLimit error.

Request instructions

Request method: GET

Request URL: https://{tls-endpoint}/DescribeHostGroupRules

Request parameters

The following table lists only the request parameters unique to this API and some common parameters. For more information, see Common Parameters.

Query

Parameter	Type	Required	Example value	Description
HostGroupId	String	Yes	c7e0e442-19bf-4fb3-b547-5992fb8b****	Host group ID.
PageNumber	Integer	No	1	Page number in paginated queries. The default is 1, which means data will be returned starting from the first page.
PageSize	Integer	No	20	Page size. The default is 20, with a maximum of 100.

Return parameters

The table below lists only the parameters unique to this API. For more information, see Return Structure.

Parameter	Type	Example value	Description
Total	Integer	1	Number of collection configurations.
RuleInfos	Array of RuleInfo	/	Collection configuration information list.

RuleInfo

Parameter	Type	Example value	Description
Paths	Array of String	["/data/nginx/log/**/access.log"]	Collection path list.
Pause	Integer	0	Collection configuration running status. 0: Running 1: Paused
RuleId	String	faf******	ID of the collection configuration.
LogType	String	delimiter_log	Collection mode. minimalist_log: Single-line full-text mode. json_log: JSON mode. delimiter_log: Separator mode. multiline_log: Multi-line full-text mode. fullregex_log: Full regular expression mode.
TopicId	String	4a************	ID of the log topic to which the collection configuration belongs.
RuleName	String	testname	Name of the collection configuration.
InputType	Integer	0	Collection type. 0: Host log file 1: K8s container standard output 2: Log files inside K8s containers
LogSample	String	2018-05-22 15:35:53.850 INFO XXXX	Log example.
TopicName	String	testname	Name of the log topic to which the collection configuration belongs.
CreateTime	String	2021-08-18 13:32:23	Collection configuration creation time.
ModifyTime	String	2021-08-18 13:32:23	Time the collection configuration was modified.
ExtractRule	Object of ExtractRule	/	Extraction rule.
ExcludePaths	Array of ExcludePath	/	Collection blacklist list.
ContainerRule	Object of ContainerRule	/	Container collection rules.
UserDefineRule	Object of UserDefineRule	/	User-defined collection rule.

ExtractRule

Parameter	Type	Example value	Description
Keys	Array of String	[ "time", "", "level", "msg" ]	List of log field names (Keys). Only valid when LogType is delimiter_log or fullregex_log. You can configure up to 100 field names. When LogType is delimiter_log, you cannot configure duplicate field names, and you cannot specify all field names as empty. When LogType is fullregex_log, you cannot configure duplicate field names, and you cannot specify field names as empty.
Quote	String	"	Quotation mark Content enclosed by quotation marks will be parsed as a single field, not separated. Valid only when LogType is set to delimiter_log.
TimeKey	String	request_time	Name of the log timestamp field. If you use the specified time field in the log as the log timestamp, you need to configure both TimeKey and TimeFormat. TimeKey and TimeFormat must be configured as a pair.
LogRegex	String	[(\d+-\d+-\w+:\d+:\d+,\d+)]\s[(\w+)]\s(.*)	The regular expression that matches the entire log. This is valid only if the collected log type is fullregex_log. The regular expression must be valid.
TimeZone	String	GMT+08:00	Time zone. Supports host time zone (default) and custom time zone. Custom time zone supports both GMT and UTC. GMT format: GMT+08:00. UTC format: Asia/Shanghai.
Delimiter	String	#	Log separator. Valid only when LogType is delimiter_log.
BeginRegex	String	[(\d+-\d+-\w+:\d+:\d+,\d+)]\s[(\w+)]\s(.*)	The regular expression required to match the first line of the log. Applies only when LogType is set to multiline_log or fullregex_log. The value must be a valid regular expression.
TimeFormat	String	%Y-%m-%dT%H:%M:%S,%f	Parsing format for the time field. If you use a specified time field in the log as the log timestamp, you need to set both TimeKey and TimeFormat. TimeKey and TimeFormat must appear together as a pair. For details on how to configure the time format, see Time Format.
LogTemplate	Object of LogTemplate	{ "Type": "Nginx", "Format": "format main '$remote_addr - $remote_user [$time_local] \"$request\" $request_time $request_length $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\"';" }	Automate log field extraction according to the specified log template.
UnMatchLogKey	String	LogParseFailed	When uploading logs with parsing failure, set the key name for those logs. UnMatchUpLoadSwitch=true and UnMatchLogKey must be used together.
FilterKeyRegex	Array of FilterKeyRegex	/	List of filter rules. When LogType is minimalist_log or multiline_log, you can configure up to 1 filter rule, and the filter field name key must be content. When LogType is set to delimiter_log, json_log, or fullregex_log, you can configure up to 5 filter rules. The filter field name key must be unique and cannot be empty. The regular expression that the log content of the filter field must match should be a valid regular expression and cannot exceed 256 characters.
TimeExtractRegex	String	[0-9]{0,2}\/[0-9a-zA-Z]+\/[0-9:,]+	Regular expression for extracting time, used to extract the time value from the TimeKey field and parse it as the corresponding collection time.
UnMatchUpLoadSwitch	Boolean	true	Upload failed parsing logs. UnMatchUpLoadSwitch=true and UnMatchLogKey must be paired together. true: upload failed parsing logs. false: do not upload failed parsing logs.
EnableNanosecond	Boolean	true	Enable nanosecond precision time. When enabled, log time parsing will include and report time with nanosecond precision. true: enable nanosecond-precision time. false: disable nanosecond-precision time.
TimeSample	String	2006-01-02T15:04:05Z07:00	Time sample. Used to determine whether the specified time parsing format is correct.

ExcludePath

Parameter	Type	Example value	Description
Type	String	Path	Collection path type. File: Filename Path: Directory
Value	String	/accesslog	Collection path. You must specify an absolute path. When Type is Path, Value represents a directory. When Type is File, Value represents a file name.

ContainerRule

Parameter	Type	Example value	Description
EnvTag	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	This button determines whether to set environment label as log tag and add it to raw log data. If enabled, TLS will add related container environment variable fields to the logs. If you set multiple key-value pairs, it means you add multiple fields. For example, set the Key to source and the Value to data_source. If the container includes the environment variable source=DC, a new field __tag__data_source__: DC will be added to the log.
Stream	String	all	Collect information. stdout: Collect the container's standard output (stdout). stderr: Collect the container's standard error (stderr). all: Collect both the container's standard output (stdout) and standard error (stderr).
KubernetesRule	Object of KubernetesRule	{ "NamespaceNameRegex": ".test.", "WorkloadType": "Deployment", "WorkloadNameRegex": ".test.", "IncludePodLabelRegex": { "Key1": "Value1", "Key2": "Value2" }, "ExcludePodLabelRegex": { "Key1": "Value1", "Key2": "Value2" }, "PodNameRegex": ".test.", "LabelTag": { "Key1": "Value1", "Key2": "Value2" } }	Collection rules for Kubernetes containers.
ContainerNameRegex	String	.test.	Name of container to collect If the container name is not specified, all containers in the host group will be collected. Supports regular expression matching. For example, if the container name is set to ^(container-test)$, all containers named container-test will be collected.
ExcludeContainerEnvRegex	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	The container environment variable denylist specifies which containers are excluded from data collection. If the denylist is disabled, all containers will be collected. If you enable the container environment variable denylist, Key is required, while Value is optional. If Value is empty, all containers whose environment variables contain the Key will be excluded from collection. If Value is not empty, only containers whose environment variables match both the Key and Value are excluded. Value supports regular expressions. For example, set Key to module and Value to ^(tcp\|udp)$. This means any container with module:tcp or module:udp in its environment variable will be excluded from collection. Multiple Key-Value pairs use a logical OR relationship. Any container that matches one of the pairs will be excluded from collection. When enabling the container environment variable denylist, Keys must not be duplicated.
IncludeContainerEnvRegex	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	Specify which containers to collect by using the container environment variable allowlist. If the allowlist is not enabled, all containers will be collected. To enable the container environment variable allowlist, enter key-value pairs. Key is required; value is optional. If Value is empty, all containers containing the Key in their environment variables will be collected. If Value is not empty, only containers with the Key in their environment variables and matching the Value will be collected. Value supports regular expression matching. For example, set Key to module and Value to ^(tcp\|udp)$. Only containers with environment variables module:tcp or module:udp will be collected. The logical relationship between multiple Key-Value pairs is logical OR; as long as a container environment variable matches any Key-Value pair, the container will be included in the collection scope. When the container environment variable allowlist is enabled, Keys can not be duplicated.
ExcludeContainerLabelRegex	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	The container label denylist is used to specify which containers should not be collected. If the denylist is disabled, all containers are collected. If the container label denylist is enabled, the Key field is required and the Value field is optional. If Value is empty, containers with the specified Key in their label will not be collected. If Value is not empty, only containers whose labels match both the Key and Value are excluded from collection. Value supports regular expressions. For example, if you set Key to app and Value to ^(test1\|test2)$, containers with the labels app:test1 and app:test2 are excluded from collection. The logic between multiple key-value pairs is logical OR. That is to say, any container whose label matches any of the key-value pairs will be excluded from collection. When the container label denylist is enabled, the Key fields must not be duplicated.
IncludeContainerLabelRegex	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	You can specify which containers to collect using container label allowlists. When the allowlist is not enabled, all containers will be collected. When enabling the container label allowlist, you need to fill in key-value pairs. Key is required; value is optional. Value is empty: Collect all containers that contain the key in their container label. When Value is not empty: Only collect containers whose Label contains the Key and whose value matches the Value. Value supports regular expression matching. For example, if you set the Key to app and the Value to ^(test1\|test2)$, only containers whose Label contains app:test1 or app:test2 will be collected. The logical relationship between multiple Key-Value pairs is OR. Any container Label that matches any Key-Value pair will be included in the collection. When enabling the container Label allowlist, keys can not be duplicated.

UserDefineRule

Parameter	Type	Example value	Description
Fields	JSON Map	{"ClusterID":"dabaad5f-7a10-4771-b3ea-d821f73e****"}	Add constant fields to logs. Constant fields must follow these restrictions: Supports uploading up to 5 constant fields. Field names (keys) must be unique and non-empty. The length must be between 1 and 128 characters, including English letters, digits, and special characters (-_./), and can not start with an underscore. Field values can not be empty and can be up to 512 KiB in size.
Plugin	Object of Plugin	{"processors":[{"json":{"field":"__content__","trim_keys":{"mode":"all","chars":"#"},"trim_values":{"mode":"all","chars":"#"},"allow_overwrite_keys":true,"allow_empty_values":true}	LogCollector plugin configuration. After enabling plugin configuration, you can add one or more LogCollector processor plugins to parse logs with complex or variable structures.
Advanced	Object of Advanced	{ "CloseInactive": 10 }	LogCollector extended configuration.
RawLogKey	String	__raw__	Original log field name. This only takes effect when EnableRawLog is set to true. RawLogKey defaults to __raw__, which means the original log data will be encapsulated in the __raw__ field and uploaded to TLS together with the parsed log data.
TailFiles	Boolean	true	The collection policy determines whether LogCollector collects incremental logs or full logs. The default is false, which means collecting all logs. true: collect incrementally. LogCollector only collects newly added content in the file while collecting logs. When new logs are written to monitored log files, it triggers LogCollector collection. For first-time collected log files, LogCollector automatically determines the collection position based on the incremental threshold TailSizeKb you specified. If the new file size does not exceed your specified incremental threshold, the first collection starts from the beginning of the new file. If the new file size exceeds your specified incremental threshold, the first collection starts at the position where the file ends minus the incremental threshold. That is to say, only incremental logs are collected. For log files collected not for the first time, LogCollector determines the collection position based on the Checkpoint and continues collection. false: (default) collect all logs. LogCollector collects logs from the beginning of each document, including historical log data.
HostnameKey	String	hostname	hostname field name. Set only when EnableHostname is true.
EnableRawLog	Boolean	true	Upload original log. true: Upload original log. false: (default) Do not upload original log.
ShardHashKey	Object of ShardHashKey	{ "HashKey": "3C" }	Rules for routing log shards. If this parameter is not set, logs are written using the default load balancing mode, and data packets are written to any available shard. Setting this parameter uses HashKey routing shard mode for log collection. TLS writes data into the shard containing the specified key value.
ParsePathRule	Object of ParsePathRule	{ "PathSample": "/data/nginx/log/dabaad5f-7a10/tls/app.log", "Regex": "\\/data\\/nginx\\/log\\/(\\w+)-(\\w+)\\/tls\\/app\\.log", "Keys": ["instance-id", "pod-name"] }	Parse rules for the collection path. After setting the rules, fields will be extracted from the collection path with the regular expressions defined in the rules and added to the log data as metadata. tip This parameter cannot be specified when collecting container standard output.
EnableHostname	Boolean	true	You can choose whether to upload the hostname field. By default, it is disabled. true: add a field in the original log to record the hostname of the log source. Specify the hostname field name using HostnameKey. false: (default) do not add the hostname field.
HostGroupLabelKey	String	host_group_label	Field name used to store host group label information
EnableHostGroupLabel	Boolean	true	You can choose whether to upload the Label information of the host group to TLS. By default, it is disabled. true: LogCollector uploads the host group Label information to the specified field. You can specify the field name in the HostGroupLabelKey parameter false (default): Do not upload host group Label information
TailSizeKb	Integer	1024	Backtracking threshold for incremental collection, measured in KiB. When the LogCollector uses incremental collection: If the size of the new log file does not exceed TailSizeKb, the collection starts from the beginning of the file. If the new log file size exceeds the TailSizeKb value, collection will start from the position after subtracting TailSizeKb from the file's end.
IgnoreOlder	Integer	7	Ignore log files not updated for the specified duration (in hours).
MultiCollectsType	String	RuleID	You can collect log files multiple times. None: Log files are uniquely identified using their log file ID, which includes the inode, device, and checksum of the first N bytes. RuleID: Log files are uniquely identified using the collection rule ID and the log file ID. TopicIDRuleName: Log files are uniquely identified using the log topic ID, the collection rule name, and the log file ID.

LogTemplate

Parameter	Type	Example value	Description
Type	String	Nginx	Type of log template. Supports the following types: Nginx: Log template for Nginx type.
Format	String	log_format main '$remote_addr - $remote_user [$time_local] "$request" $request_time $request_length $status $body_bytes_sent "$http_referer" "$http_user_agent"';	Log template format.

FilterKeyRegex

Parameter	Type	Example value	Description
Key	String	__content__	Name of the filter field.
Regex	String	.ERROR.	The log content in the filter field must match the specified regular expression.

KubernetesRule

Parameter	Type	Example value	Description
LabelTag	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	This button determines whether to add Kubernetes Labels as log tags to the raw log data. If enabled, TLS will add Kubernetes Pod Label related fields to the logs. If you set multiple key-value pairs, multiple fields are added. For example, set the key to source and the value to data_source. If the Pod has the label source=DC, TLS will add the field __tag__data_source__: DC to the log.
PodNameRegex	String	.test.	Pod name specifies the container to collect. If you do not specify a Pod name, all containers will be collected. Pod name supports regular expressions. For example, set the Pod name to ^(http.*)$ to collect all containers under Pods whose names start with "http."
WorkloadType	String	Deployment	Specify the container for collection based on workload type. Only one type can be selected. If no type is specified, containers of all types are collected. Supported workload types: Deployment: Stateless workload StatefulSet: Stateful workload DaemonSet: Daemon process Job: Task CronJob: Scheduled SQL task
AnnotationTag	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	This button determines whether to set Kubernetes Annotation to log tags and add it into the raw log data. If enabled, TLS will add Kubernetes Pod annotation fields to the logs. If you set multiple key-value pairs, multiple fields are added. For example, if you set the key to sink and the value to data_sink, and the Pod includes the annotation sink=ck, a new field __tag__data_sink__: ck will be added to the log.
WorkloadNameRegex	String	.test.	By configuring the workload name, you can specify the container to collect. If no workload name is specified, all containers are collected. Workload names support regular expression matching. For example, setting the workload name to ^(http.*)$ collects all containers under workloads that start with http.
NamespaceNameRegex	String	.test.	Name of the Kubernetes namespace to be collected. If no Namespace is specified, all containers will be collected. Namespace names support regular expression matching. For example, setting the namespace name to ^(tcp\|udp)$ means collecting all containers in the tcp and udp namespaces.
ExcludePodLabelRegex	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	Use the Pod Label denylist to specify which containers to exclude from collection. If not enabled, all containers will be collected. If you need to set a Pod Label denylist, the Key field is required, while the Value field is optional. If the Value field is empty, all containers with the specified Key in the Pod Label will be excluded from collection. If the Value field is not empty, only containers with the specified Key and a Value that matches will be excluded from collection. The Value field supports regular expressions. For example, if you set Key to module and Value to ^(tcp\|udp)$, containers with module:tcp and module:udp in their Pod Label will be excluded. tip The logic between multiple Key-Value pairs is logical OR; as long as any Key-Value pair is matched, the container will be excluded from collection. When enabling the Pod Label denylist, keys can not be duplicated. You must add .* for regular expression fuzzy matching. For example, ^(http.*)$ matches strings that start with "http".
IncludePodLabelRegex	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	The Pod Label allowlist is used to specify the containers to be collected. If the Pod Label allowlist is not enabled, all containers will be collected. If you need to set the Pod label allowlist, the key field is required. If the value is empty, all containers with the key in the Pod label are collected. If the value is not empty, only containers that match the key and the value in Pod Label are collected. Regular expressions are supported. The value supports regular expressions. For example, set the key to module and set the value to ^(tcp\|udp)$. Only containers with the Pod label module:tcp or module:udp will be collected. tip The logical relationship between multiple key-value pairs is logical OR. As long as the Pod label matches any key-value pair, it will be included in the collection. When enabling the Pod label allowlist, duplicate keys are not allowed. For fuzzy regular expression matching, you must add .. For example, ^(http.)$ matches any string that starts with "http".
EnableAllLabelTag	Boolean	true	This button determines whether to add all Kubernetes labels as log tags to the raw log data. When enabled, TLS will add all related Kubernetes Pod label fields to logs. For example, if a Pod contains the labels source=DC and destination=CS, TLS will add the fields __tag__source__: DC and __tag__destination__: CS to the logs.
ExcludePodAnnotationRegex	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	Pod Annotation denylist specifies containers that are excluded from collection. If you need to set a Pod Annotation denylist, the Key is required and the Value is optional. If the Value is empty, all containers in Pods with the specified Key in their Pod Annotation will be excluded. If the Value is not empty, only containers in Pods with the specified Key and a Value matching the specified Value in their Pod Annotation will be excluded. By default, Value uses string matching. Containers only match if the value is exactly the same as the value of the Pod annotation. If the value starts with^and ends with$, it uses regular expression matching. For example, set the Key as "app" and set the Value as ^(test1\|test2)$. This matches containers in Pods whose annotation contains app:test1 or app:test2. tip The logical relationship between multiple Key-Value pairs is logical OR, which means that a Pod will be included for collection as long as it meets any pair. When enabling Pod Annotation denylist, duplicate Keys are not allowed. When using regular expression fuzzy matching, you must add .. For example, ^(http.)$ matches values that start with "http".
IncludePodAnnotationRegex	JSON Map	{ "Key1": "Value1", "Key2": "Value2" }	Pod annotation allowlist is used to specify the containers to be collected. If you need to set a Pod annotation allowlist, the Key field is required and the Value field is optional. If the Value field is empty, all containers in Pods with the specified Key in their annotations will match. If the Value field is not empty, containers in Pods whose annotation contains the Key and the Key's value matches the Value will match. By default, Value uses string matching. Containers only match if the value is exactly the same as the value of the Pod annotation. If the value starts with^and ends with$, it uses regular expression matching. For example, set the Key as "app" and set the Value as ^(test1\|test2)$, and then the containers in Pods whose annotation contains app:test1 or app:test2 will match. tip The logical relation between multiple Key-Value pairs is logical OR, which means Pods are included in the collection scope as long as any pair matches their annotation. When the Pod Annotation allowlist is enabled, the Key must not be duplicated. You must add .* for regex fuzzy matching. For example, ^(http.*)$ matches characters that start with "http".

Plugin

Parameter	Type	Example value	Description
processors	Array of JSON Map	{ "processors":[ { "json":{ "field":"__content__", "trim_keys":{ "mode":"all", "chars":"#" }, "trim_values":{ "mode":"all", "chars":"#" }, "allow_overwrite_keys":true, "allow_empty_values":true } } ] }	LogCollector plugin. For the list of supported plugins and parameter instructions, see LogCollector Plugin Overview.

Advanced

Parameter	Type	Example value	Description
CloseEOF	Boolean	true	After reading to the end of the log file, determine whether to release the handle of the log file. The default value is false.
CloseRemoved	Boolean	true	Release the handle for the log file after it is removed. The default value is false.
CloseRenamed	Boolean	true	Release the log file handle after it is renamed. The default value is false.
CloseTimeout	Integer	1800	LogCollector monitors the maximum duration of log files. Measured in seconds, with a default of 0 seconds, meaning LogCollector does not limit the monitoring duration of log files. LogCollector starts timing when it begins monitoring a log file. After exceeding the specified duration, the log file handle is released immediately and monitoring ends, regardless of whether the file has been fully read.
CloseInactive	Integer	10	Wait time for releasing the log file handle. If no new logs are written after the specified time, the log file handle is released. The unit is seconds, ranging from 1 to 300 seconds, with a default value of 60 seconds.
NoLineTerminatorEOFMaxTime	Integer	5	Maximum wait time when LogCollector does not detect a newline character. Unit: seconds. Default is 5s. LogCollector starts timing when it begins reading the log file content. If no newline character is detected within the specified time, the logs in the buffer will be sent. If the file write interval is long, a complete log entry may be split into two parts and written separately. You can adjust this parameter based on your log write interval.

ShardHashKey

Parameter	Type	Example value	Description
HashKey	String	3C	HashKey of the log group, used to specify the shard where the current log group is written. The value range for this parameter is [00000000000000000000000000000000-ffffffffffffffffffffffffffffffff).

ParsePathRule

Parameter	Type	Example value	Description
Keys	Array of String	["instance-id","pod-name"]	List of field names. The TLS parses the sample path (PathSample) into multiple fields using regex. Keys specify the names of each field. You can configure up to 100 field names. Field names can not be empty or duplicated.
Regex	String	/var/logs/([a-z])_any_([a-z])/test.log	Regular expression used to extract path fields. It must match the sample collection path; otherwise, extraction fails.
PathSample	String	/var/logs/instanceid_any_podname/test.log	Collection path sample from an actual scenario The collection path sample must be an absolute path The path sample can not contain wildcards、?、*。

Sample request

Sample response

Error Code

The following table lists the error codes related to the business logic of this API. For public error codes, see the Public Error Codes documentation.

HTTP status code	Error Code	Error message	Description
400	InvalidArgument	Invalid argument key %s, value %s, please check argument.	Invalid parameter.
404	HostGroupNotExist	HostGroup %s does not exist	Host group does not exist
500	InternalServerError	We encountered an unexpected server error, please try again later.	Internal server error.

Torch Log Service