Docs

Torch Log Service

Query and analysis FAQ

How to improve the speed of search analysis?

This paper introduces a method to improve the speed of log retrieval analysis.

Increase the Number of Shards

Increasing the number of shards can enhance the read and write capabilities of the logging service, but only for newly written data. Shard represents computing resources. The more shards, the faster the computation. You can increase the number of Shards appropriately by splitting them. For specific operation, see Split shard.

explain

Increasing the number of Shards will incur more charges, please increase as needed. For shard's billing instructions, see Billing items based on pay-by-feature mode.

Narrow down the time frame or amount of data to retrieve

The larger the retrieval time range, the slower the retrieval speed. It is recommended to appropriately narrow the retrieval time range to improve the retrieval speed.

The larger the amount of retrieved data, the slower the retrieval speed. It is recommended to reduce the amount of retrieved data appropriately to improve the retrieval speed.

Repeated search operation

When the search results are inaccurate, you can attempt to repeat the search operation multiple times. Each time it is retrieved, the underlying acceleration mechanism of the log service retrieves based on existing results, so repeated retrievals can improve the accuracy of the retrieved results. For more information, see How to handle inaccurate results.

optimized search statement

Sentences that take a long time to retrieve have the following characteristics.

There are many search conditions.

The amount of data retrieved and scanned is large.

You can optimize the retrieval analysis statement by doing the following.

Remove unnecessary search conditions

For example, when using multi-condition retrieval, if the log that satisfies condition A must satisfy condition B, then condition B can be removed. Examples are as follows:

Search statements (fast)

log_msg: "INFO| write logs sucessful"

Search statements (slow)

log_msg: "INFO| write logs sucessful" AND log_level:INFO

Merge the same Key Range search criteria.

When using range retrieval, replace the single-range operator with a two-range operator if you need to limit the start and end range of the retrieval. Examples are as follows:

Search statements (fast)

log_status: [1000 TO 2000}

Search statements (slow)

log_status: >= 1000 AND log_status: < 2000

Put high-hit filter criteria into SQL statements.

In the scenario where retrieval and analysis are used together, the condition of high hit rate can be placed in the SQL section to improve the retrieval rate. For example, in the following example, if the ratio of the number of logs to the total number of logs that meet log_level: INFO exceeds 50%, the condition will be filtered into the SQL statement, which is faster.

Retrieve analysis statements (fast)

* | select count(*) as cnt where log_level = 'INFO'

Retrieve parsing statements (slow)

log_level:INFO | select count(*) as cnt

Specify the columns to be acquired in the SQL analysis statement to avoid full column reads.

If you need to get all the columns, use the search syntax. Examples are as follows:

Retrieve analysis statements (fast)

* | select a, b, c

Retrieve parsing statements (slow)

* | select *

Last updated: 2026.06.25 17:01:03