Torch Log Service
Parameter | Type | Required | Example value | Description |
Parameter | Type | Required | Example value | Description |
ProjectId | String | No | f3e901c3-b17f-42fd-aa8c-dc91a6c7**** | Specify the project ID. Used to query log shipping configurations under the specified project. |
ProjectName | String | No | fa************ | Specify the project name. Used to query log shipping configuration under the specified project. Support fuzzy match. |
IamProjectName | String | No | default | Specify the IAM project name. Used to query log shipping configurations under the specified IAM project. |
PageNumber | Integer | No | 10 | Page size. Default is 20, maximum is 100. |
PageSize | Integer | No | 20 | Page number when performing a paginated query. By default, this is 1, which means data is returned starting from the first page. |
ShipperName | String | No | test | Specify the log shipping configuration name; supports fuzzy matching. |
ShipperId | String | No | f3e901c3-b17f-42fd-aa8c-dc91a6c7**** | Specify the log shipping configuration ID. |
TopicName | String | No | test | Specify the log topic name. Used to query the log shipping configuration for this log topic. Fuzzy matching supported. |
TopicId | String | No | f3e901c3-b17f-42fd-aa8c-dc91a6c7**** | Specify the log topic ID. Used to query log shipping configurations related to this log topic. |
ShipperType | String | No | tos | Specify the log shipping type to query its related log shipping configuration. Available options include:
|
Parameter | Type | Example value | Description |
Parameter | Type | Example value | Description |
Shippers | / | Information related to delivery configuration. | |
Total | Integer | 5 | Number of log shipping configurations. |
Parameter | Type | Example value | Description |
Parameter | Type | Example value | Description |
Status | Boolean | true | Enable shipping configuration. |
TopicId | String | c7**** | ID of the log topic containing the log to be shipped. |
ProjectId | String | dc7**** | ID of the project where the log to be shipped is located. |
ShipperId | String | 4a9bd4bd-53f1-43ff-b88a-64ee1be5**** | Log shipping configuration ID. |
TopicName | String | mytopic | Name of the log topic where the logs to be shipped are stored. |
CreateTime | String | 2024-08-01 14:01:02 | Creation time of the log shipping configuration. |
ModifyTime | String | 2024-08-01 14:01:10 | Last modified time of the shipping configuration. |
ContentInfo | / | Content format configuration for log delivery. | |
DashboardId | String | / | The default built-in dashboard ID for shipping. |
ProjectName | String | myproject | Name of the project where the logs to be shipped are located. |
ShipperName | String | tosship | Delivery configuration name. |
ShipperType | String | tos | Shipping type. |
ShipperEndTime | Integer | 1723709712496 | Log shipping end time. |
TosShipperInfo | / | Information delivered to TOS | |
KafkaShipperInfo | / | Information delivered to Kafka. | |
ShipperStartTime | Integer | 1723709602496 | Log shipping start time. |
RoleTrn | String | trn:iam::2100000001:role/TLSETLAccessForUserA | Trn for custom roles. |
Parameter | Type | Example value | Description |
Parameter | Type | Example value | Description |
Format | String | json | Log content parsing format.
|
CsvInfo | / | CSV format log content configuration. | |
JsonInfo | / | Log content configuration in JSON/JSONL format. |
Parameter | Type | Example value | Description |
Parameter | Type | Example value | Description |
Bucket | String | mytos | Name of the object storage bucket. |
Prefix | String | testprefix | Name of the top-level directory in the bucket. All log data shipped by this configuration will be stored under this directory. |
MaxSize | Integer | 5 | Maximum raw file size that can be shipped per shard (Shard), which refers to the uncompressed log file size Measured in MiB |
Compress | String | snappy | Compression format. |
Interval | Integer | 300 | Log shipping time interval (in seconds). |
PartitionFormat | String | %Y/%m/%d/%H/%M | Shard rules for log shipping logs. |
Parameter | Type | Example value | Description |
Parameter | Type | Example value | Description |
EndTime | Integer | 1723709712496 | End time for log shipping, in milliseconds. If not set, shipping continues indefinitely tip When shipping to Kafka, it is recommended to set the shipping time range using the ShipperStartTime and ShipperEndTime fields, and leave the StartTime and EndTime fields blank |
Compress | String | snappy | Compression format; currently supports snappy, gzip, lz4, and none. |
Instance | String | kafka-cnngbnntswg1**** | Kafka instance. |
StartTime | Integer | 1723709602496 | Delivery start time in milliseconds. If not specified, the default is the current time. tip When delivering to Kafka, use the ShipperStartTime and ShipperEndTime fields to set the delivery time range, and leave the StartTime and EndTime fields empty. |
KafkaTopic | String | topic-c | Kafka Topic name. All log data delivered by this configuration will be sent to this Topic. |
Parameter | Type | Example value | Description |
Parameter | Type | Example value | Description |
Keys | Array of String | ["user","log","ip"] | Configure the fields that need to be shipped.
|
Delimiter | String | , | Separator: Supports comma, tab, vertical bar, semicolon, and space. |
EscapeChar | String | false | Escape character: When the field contains a separator, use an escape character to wrap the field. Currently, only single quotes, double quotes, and empty characters are supported. |
PrintHeader | Boolean | false | Print the Key in the first row. |
NonFieldContent | String | test | Content for invalid field fill, length 0–128 |
Parameter | Type | Example value | Description |
Parameter | Type | Example value | Description |
Keys | Array of String | ["__content", "__pod_name__"] | When using JSON/JSONL format for shipping, if this parameter is not configured, all fields will be shipped, including __content__ (required), __source__, __path__, __time__, __image_name__, __container_name__, __pod_name__, __pod_uid__, namespace, __tag____client_ip__, and __tag____receive_time__. |
Enable | Boolean | true | Enable flag. |
Escape | Boolean | true | Escape characters supported. This must be set to true. |