Torch Log Service
Parameter | Type | Required | Example value | Description |
Parameter | Type | Required | Example value | Description |
ContentInfo | Yes | / | Configuration for the delivery format of log content. | |
KafkaShipperInfo | No | / | Configuration for delivering to Kafka. | |
ShipperEndTime | Integer | No | 1723709712496 | Delivery end time, in milliseconds. If not set, delivery will continue indefinitely. tip When delivering to Kafka, it is recommended to configure the time range for delivery using the ShipperStartTime and ShipperEndTime fields, and leave the StartTime and EndTime fields empty. |
ShipperName | String | Yes | tosship | |
ShipperStartTime | Integer | No | 1723709602496 | Delivery start time, in milliseconds timestamp. If not configured, the default is the current time. tip For delivery to Kafka, it is recommended to use the ShipperStartTime and ShipperEndTime fields to set the time range. Leave the StartTime and EndTime fields blank. |
ShipperType | String | Yes | tos | Delivery type.
|
TopicId | String | Yes | c7**** | ID of the log topic containing logs to be delivered. |
TosShipperInfo | No | / | Related settings for delivery to TOS object storage. | |
RoleTrn | String | No | trn:iam::2100000001:role/TLSETLAccessForUserA | Custom role Trn. |
Parameter | Type | Required | Example value | Description |
Parameter | Type | Required | Example value | Description |
Format | String | No | json | Log content parsing format.
|
CsvInfo | No | / | CSV format log content configuration. | |
JsonInfo | No | / | Configure log content in JSON/JSONL format. |
Parameter | Type | Required | Example value | Description |
Parameter | Type | Required | Example value | Description |
EndTime | Integer | No | 1723709712496 | Delivery end time, in milliseconds. If not set, delivery will continue indefinitely. tip When sending to Kafka, use the ShipperStartTime and ShipperEndTime fields to specify the delivery time range, and leave the StartTime and EndTime fields empty. |
Compress | String | Yes | snappy | Compression format. Currently supports snappy, gzip, lz4, and none. |
Instance | String | Yes | kafka-cnngbnntswg1**** | Kafka instance. |
StartTime | Integer | No | 1723709602496 | Delivery start time in milliseconds. If not configured, the current time is used by default. tip When delivering to Kafka, we recommend using the ShipperStartTime and ShipperEndTime fields to specify the delivery time range, and leaving the StartTime and EndTime fields empty. |
KafkaTopic | String | Yes | topic-c | Kafka Topic name All log data delivered through this configuration will be sent to the specified Topic |
Parameter | Type | Required | Example value | Description |
Parameter | Type | Required | Example value | Description |
Bucket | String | Yes | mytos | Select a TOS bucket in the same region as the source log topic.
|
Prefix | String | No | testprefix | Top-level directory name of the bucket. All log data delivered by this configuration will be stored in this directory.
|
MaxSize | Integer | No | 5 | The maximum size of the original file that can be delivered to each shard, referring to the uncompressed log file size. The unit is MiB, and the value range is 5 to 256. |
Compress | String | No | snappy | Compression format: currently supports snappy, gzip, lz4, and none. |
Interval | Integer | No | 300 | Delivery interval in seconds. Valid range: 300–900. |
PartitionFormat | String | No | %Y/%m/%d/%H/%M | Shard rules for log delivery.
|
Parameter | Type | Required | Example value | Description |
Parameter | Type | Required | Example value | Description |
Keys | Array of String | Yes | ["user","log","ip"] | Configure the fields to be delivered.
|
Delimiter | String | Yes | , | Separator: supports commas, tabs, vertical bars, semicolons, and spaces. |
EscapeChar | String | Yes | false | Escape character: When a field contains a separator, use an escape character to enclose the field; currently, only single quotes, double quotes, and empty characters are supported. |
PrintHeader | Boolean | Yes | false | Print the key in the first row |
NonFieldContent | String | Yes | test | Invalid field fill content, length: 0–128 |
Parameter | Type | Required | Example value | Description |
Parameter | Type | Required | Example value | Description |
Keys | Array of String | No | ["__content", "__pod_name__"] | When submitting in JSON/JSONL format, if this parameter is not set, all fields will be delivered, including __content__ (required), __source__, __path__, __time__, __image_name__, __container_name__, __pod_name__, __pod_uid__, namespace, __tag____client_ip__, and __tag____receive_time__. |
Enable | Boolean | Yes | true | Enable flag. |
Escape | Boolean | No | true | Enable escaping. Must be set to true. |
Parameter | Type | Example value | Description |
Parameter | Type | Example value | Description |
ShipperId | String | 4a9bd4bd-53f1-43ff-b88a-64ee1be5**** | Delivery configuration ID. |