Docs

Torch Log Service

Index configuration FAQ

How to configure indexes for logs

When configuring indexes for JSON logs, you can refer to the following configuration approach and examples.

Options of index types

TLS provides two types of index configurations: Full-text index and Key-value index. Refer to the following notes to select the appropriate index type. For more information, see Configure indexes.

Notice

If you configure both full-text and key-value indexes, the key-value index configuration will take precedence for fields set with key-value indexing.

Full-text index: If you need to query all fields in the logs, create a full-text index.

Key-value index: If you only need to query specific fields, create a Key-value index for those fields to reduce indexing costs.

If you need SQL analysis on a field, you must create a key-value index for the target field and enable the statistics feature. For example, if you only analyze the status and method fields, you only need to configure key-value indexes for these two fields and enable the statistics feature.

Options of data types for indexed fields

The data types supported for indexed fields in TLS include text, long, double, and JSON. For more information, see Index data types.

When selecting the appropriate data type for the target field, you can refer to the following guidelines:

Non-standard JSON format: If the field value contains JSON-like content but is not in standard JSON format, we recommend that you set it as text type.

Standard JSON format: If the field value is in standard JSON format, we recommend that you set it as JSON type. When set to JSON type, TLS will automatically expand the JSON content.

When set to JSON type, TLS supports parsing valid parts of content that is not fully compliant with the JSON format.

After setting the type to JSON, you can further analyze subfields within a JSON object by creating separate indexes for each subfield. Index data types can be text, long, or double. After configuration, query and analysis can be precise to the subfield level. Query and analysis speed for subfields is improved. But additional index fees will generate.

Notice

TLS supports indexing subfields within JSON objects, but it is not recommended to index subfields that contain subfields themselves. If you index a nested JSON object within a JSON object, TLS will serialize the JSON value of the subfield as a string.

TLS does not support indexing fields with JSON array values, nor does it support indexing fields within a JSON array.

Configuration example

Log example

The logging field and process_info field are both of JSON type.

Query and analysis requirements

Count log entries by each log level using logging.level field.

Count the number of logs output per minute using the process_info.delivered field.

Count the number of failed log entries for each processing task within a specified time range using task_id and process_info.failed fields.

Index configuration instructions

Field	Description
logging	The logging field uses JSON format. Set the index data type to JSON and enable statistics. "logging.level" refers to the log level. Set the index data type to text and enable statistics. "logging.message" is rarely queried. Therefore, separate index is not required. When analysis is needed, use JSON-related functions. For details on JSON functions, see JSON functions.
process_info	The process_info field uses JSON format. Set the index data type to JSON and enable statistics. "process_info.faild" refers to the number of failed logs output during the processing task. Set the index data type to long and enable statistics. "process_info.delivered" refers to the output count during the processing task. Set the index data type to long and enable statistics.
task_id	The task_id field refers to the processing task ID. Set its index data type to text and enable statistics.

Index configuration example

Last updated: 2026.06.25 16:58:10