This topic lists the query syntax supported by TLS query features.

Notice

Syntax operators are case-insensitive. For example, both AND and and can be identified as query logical operators.

For logical operations (AND, OR, &&, !, etc.), half-width parentheses (()) is required to define their logical precedence. Otherwise, search results may differ from expectations. For example, to search for logs on a specific machine under the log directory that does not contain error, set the query condition as ((__source__:192.168.10.10 AND __path__:/var/log/*) NOT error).

If space is used to separate multiple query statements, it indicates the logical "AND". That is to say, the space is equivalent to AND. For example, warning error is equivalent to warning AND error. This syntax means retrieving logs containing both keywords warning and error.

Operators in the query syntax are reserved characters in TLS. If any of these reserved characters are included in the query keywords, escape characters are required.

Query methods

TLS supports full-text and key-value queries. The details are shown in the table below.

Query methods	Description
Full-text query	When entering query statements, simply type the keywords without specifying log field names. TLS automatically splits the raw log by the specified delimiter. If the segmented log contains a word that matches the query criteria, it is a match. The query statement format is as follows: keywords1 [ [ and \| or \| not ] keywords2 ] ... In this case, keywords can be replaced with words that need to be queried. Logical operators can be included. For example, 404 AND 500 queries all logs containing both the 404 and 500 keywords. Notice You need to enable the full-text index before performing full-text query. In query statements, if the query value contains special characters (such as spaces, Chinese characters, etc.), query operators (and, or, etc.), or delimiter, enclose it in double quotes (""). For example, "cn guilin".
Key-value query (Key:Value)	When entering query statements, you need to specify both the field name and field value. The query matches if the value of the specified field in the log meets the specified conditions. Query statement format: key1 [ : \| > \| >= \| < \| <= \| = ] value1 [ [ and \| or \| not ] key2 [ : \| > \| >= \| < \| <= \| = ] value2 ]... In this case, key refers to the index field. For example, status:500 queries access logs where status is 500. Notice In query statements, if a field name or field value contains special characters (such as spaces, Chinese characters, etc.), query operators (and, or, etc.), or delimiter, enclose it in double quotes (""). For example, "cn guilin". Only long field and double field support numeric range queries. You need to enable the key-value index before performing key-value queries.

Query accuracy

TLS supports exact query and fuzzy query. The details are shown in the table below.

Query accuracy	Description	Example
Precise query	If complete keyword is specified during query, query results will match based on complete keyword. For example, if the query condition is GET, logs containing the keyword GET will be retrieved. When TLS queries data, it uses delimiter. Results from exact queries may not fully match the query condition. For example, when executing the query statement successful request, TLS queries based on successful and request. Therefore, logs containing successful or request will be retrieved, instead of the exact match of successful request. For exact match of phrase successful request, use phrase queries or the LIKE syntax. For details, see How to accurately query phrases?.	region: cn-shanghai: Queries logs where the region field contains cn-shanghai. URL:#"successful request": Queries logs containing the phrase successful request. * \| SELECT * WHERE "__content__" LIKE '%Error 2024-06-24%': Queries logs where the __content__ field contains the phrase Error 2024-06-24.
Fuzzy query	If you specify only part of a keyword in query statement (add wildcard asterisk * or question mark ?), TLS matches up to 100 words and returns logs that contain these 100 words and meet the query conditions. For example, the search criteria erro* queries the first 100 words starting with "erro" in all logs, and then returns logs that contain these 100 words and meet the query conditions. For details, see Instructions for use. The more precise the specified word, the more precise the query results. An asterisk () or question mark (?) can only appear in the middle or at the end of a keyword, not at the beginning. If asterisk () or question mark (?) are enclosed in double quotes (" "), asterisk () or question mark (?) represents only literal characters and can not be used for fuzzy match. Fields of numeric types (long, double) do not support fuzzy query using asterisk () or question mark (?). You can query by specifying a numeric range, such as age:{20 TO 30}.	TopicID:6d67: Queries logs that meet two conditions: contain field TopicID; field value starts with 6d and ends with 67. cn: Queries logs containing words that start with cn.

Operators

The table below lists the query operators supported by TLS.

Operator	Description	Example
AND	Logical operator "AND". It is used to acquire the intersection of multiple search results. If multiple logical operations are specified, use half-width parentheses () to define the priority.	__path__:\/var\/log\/* AND error
OR	Logical operator "OR" returns the union of multiple query results. If multiple logical operations are specified, use half-width parentheses () to define the priority.	error OR warning
NOT	Logical operator NOT. Using it means that query results do not include keywords following NOT. If multiple logical operations are specified, use half-width parentheses () to define the priority.	ERROR AND NOT __path__:\/var\/log\/*
TO	Range logical operator TO. Note: a space should be added before and after TO. If the starting value is greater than the ending value, it will be automatically replaced. age:[20 TO 1] is equivalent to age:[1 TO 20].	age:[1 TO 20]
''	Single quotes: When a string inside single quotes contains preset delimiters, the keywords split by the delimiters are connected by an AND relationship. The asterisk * and question mark ? wrapped in single quotes ' represent wildcards.	'today is friday'
""	Double quotes: When a string inside double quotes contains preset delimiters, the keywords split by the delimiters are connected by an AND relationship. The asterisk * and question mark ? enclosed in double quotes " are not wildcards but literal characters.	__hostname__:"Jone Doe"
:	Colon: refers to key-value query; returns logs containing the specified key and value.	level:info
	Wildcard, used for fuzzy queries. The wildcard * can match zero, single, or multiple characters. If asterisk () or question mark (?) are enclosed in double quotes (" "), asterisk () or question mark (?) represents only literal characters and can not be used for fuzzy match. For example: gu* matches keywords that start with gu. "gu" matches the keyword gu.	level:erro*
?	Wildcard, used for fuzzy queries. The wildcard ? can match a single character If asterisk () or question mark (?) are enclosed in double quotes (" "), asterisk () or question mark (?) represents only literal characters and can not be used for fuzzy match. For example: gu?n matches the keyword that starts with gu, ends with n, and has one character in between. "gu?n" matches the keyword gu?n	level:erro?
()	Grouping operators. It is used to control the priority among multiple logical operations.	((__source__:192.168.10.10 AND __path__:\/var\/log\/*) NOT error)
=	Equal sign: query logs with the specified value	age=18
!=	Inequality sign: query logs that do not contain the specified value	age!=18
>	Range operator: query logs where the value in a numeric field is greater than a specific number.	age>18
>=	Range operator. Used to query logs where the value in a numeric field is greater than or equal to a specific number	age>=18
<	Range operator. Used to query logs where the value in a numeric field is less than a specified value.	age<18
<=	Range operator. Used to query logs where the value in a numeric field is less than or equal to a specified value.	age<=18
[]	Range operator. The boundary values are included in the range.	age:[20 TO 30]
{}	Range operator. Boundary values are not included within range.	age:{20 TO 30}
\	Escape character. If the query keywords contain reserved characters, you need use the escape character to process them before querying. An escaped character represents itself, not a keyword. Characters that need to be escaped include + - = & \| > < !. ( ) { } [ ] ^ " ~ * ? : \ /. Notice When using double quotes for queries, only escape " \\. When using single quotes for queries, only escape ' \\ * ?.	__path__:\/images\/log

Reference for query statements

Common query statements

Scenarios	Statement
Query logs containing the keyword ERROR.	ERROR
Query logs containing the level field.	level:*
Query logs that do not contain the level field.	NOT level:*
Query logs where the level field is not empty	NOT level:""
Query logs where the level field is empty	level:""
Query logs where the level field is not null.	NOT level:"null"
Query logs from a specified IP address or multiple machines.	__source__:192.168.10.10 or __source__:192.168.10.*
Query logs from files with a specific name in the log collection path.	__path__:"/var/log/access.log" or __path__:/var/log/*.log
Query access logs whose status code is 4xx.	status:[400 TO 500}
Query logs with status codes greater than 400, i.e., logs that failed to access.	status:>400
In access logs that GET requests, query logs of failed requests.	method:GET AND status:>400
Query exception logs, i.e., logs not at INFO level.	NOT level:INFO
Query logs at ERROR level or WARNING level.	level:ERROR OR level:WARNING
Query exception logs from the specified machine, i.e., logs from the specified IP address with a level other than INFO.	__source__:192.168.10.10 NOT level:INFO
Logs from the specified IP address with a level of ERROR or WARNING.	__source__:192.168.10.10 AND (level:ERROR OR level:WARNING)
Query exception logs in the specified file on the designated machine, i.e., logs from the specified IP address and specified file path with a level other than INFO.	(__source__:192.168.10.10 AND __path__:\/var\/log\/access.log) NOT level:INFO
Query logs for the specified pod name using the reserved field __pod_name__.	__pod_name__:nginx
Query logs where the Pod name is application and the request latency exceeds 3 seconds.	__pod_name__:application AND request_time:>3
Query logs that meet two conditions: contain a field named ProjectID; filed value starts with 6d and ends with 67.	ProjectID:6d*67
Query logs containing the keyword not.	"not" Notice not is used as a default TLS query operator. If not is just a regular query string, use double quotation marks ("") to enclose it.

Query based on delimiter

If you enable the delimiter feature when creating an index, TLS splits the log content into multiple keywords based on the delimiter. You can query logs by each keyword. TLS supports delimiters including !@#%^&*"()-_=', <>/?|;:\n\t\r[]{}\.

For example, provides that the region field value is cn-guilin, and you have configured a Key-value index for this field.

If the delimiter feature is not enabled, cn-guilin is identified as a whole. Using guilin query statement, the log can not be queried.

If you set the delimiter to !@#%^&*"()-_=', <>/?|;:\n\t\r[]{}\, cn-guilin will be split into cn and guilin. You can refer to example statements below for log queries.

Scenarios	Query statements
Query logs where the value of the region field contains guilin.	region:guilin
Query logs where the value of the region field contains cn and guilin.	region:"cn guilin"
Query logs containing words that start with gu.	gu*

Phrase query

Phrase queries allow you to precisely match specified phrases. For details, see Query logs via phrase query.

Scenarios	Query statements
Query logs where the value of the URL field contains ShardId=7.	URL:#"ShardId=7"

Reserved field query

Reserved field	Description	Example
__source__	Filter logs from the specified IP address. Supports fuzzy query using wildcards.	__source__:127.0.0.*
__path__	Filter logs from files in the specified path. Supports fuzzy query using wildcards.	__path__:\/var\/log\/messages
__time__	Filter logs using the __time__ field in the log. Range queries are supported.	__time__:[1524952304123 TO 1724958207123]

Example queries

The following is an example that shows how to use query syntax from various perspectives, including full-text query, key-value query, exact query, fuzzy query, phrase query, and SQL LIKE query.

raw logs

Configure indexes

Different index configurations result in different query and analysis outcomes. When both full-text and query indexes are created, the key-value index configuration applies.

Query statements

Fuzzy query

Category	Scenarios	Query statements and results
Basic fuzzy query	Query logs containing keywords that start with Consume.	Query statements Consume* Query result Return logs containing keywords such as ConsumeOriginalLogs, ConsumeKafkaLogs, and ConsumerHeartbeat.
Fuzzy query using SQL LIKE syntax	Query logs where the host field contains the string ivolces.	Query statements \| SELECT * WHERE Host LIKE '%ivolces%' Query result

Precise query

Category	Scenarios	Query statements and results
Basic exact query	Query logs containing the keyword ConsumeLogs.	Query statements ConsumeLogs Query result Return logs containing the ConsumeLogs keyword.
		Query logs in the URL field that contain ConsumeLogs.	Query statements URL:ConsumeLogs Query result Return logs in the URL field that contains the ConsumeLogs keyword.
		Query logs related to ConsumeLogs?ShardId=5.	In this example, the question mark (?) and equals sign (=) are delimiters. Thus, TLS splits ConsumeLogs?ShardId=5 into ConsumeLogs, ShardId, and 5 for querying. If you need an exact match for the phrase ConsumeLogs?ShardId=5, refer to the following phrase search and SQL LIKE query. Query statements "ConsumeLogs?ShardId=5" Query result Return all logs that contain ConsumeLogs, ShardId, and 5 at the same time.
Phrase query	Precisely query logs related to ConsumeLogs?ShardId=5.	To match ConsumeLogs?ShardId=5 exactly, you can use phrase queries or precise queries based on SQL LIKE syntax. Query statements #"ConsumeLogs?ShardId=5" Query result Exactly match logs related to the phrase ConsumeLogs?ShardId=5.
Precise query based on SQL LIKE syntax	Precisely query logs where the URL field contains ConsumeLogs?ShardId=5.	Query statements \| SELECT * WHERE URL LIKE '%ConsumeLogs?ShardId=5%' Query result Exactly match logs related to the phrase ConsumeLogs?ShardId=5.

Torch Log Service