You need to enable JavaScript to run this app.
Torch Log Service

Torch Log Service

Copy page
Download PDF
Data shipping API
DescribeShippers
Copy page
Download PDF
DescribeShippers
Call DescribeShippers to retrieve the log shipping configuration list.​
Request instructions​
  • Request method: GET​
  • Request URL: https://{tls-endpoint}/DescribeShippers​
Request parameters​
The table below lists unique parameters for this API and some common parameters. For more details, see Common Parameters.​
Query​
​
Parameter​
Type​
Required​
Example value​
Description​
ProjectId​
String​
No​
f3e901c3-b17f-42fd-aa8c-dc91a6c7****​
Specify the project ID. Used to query log shipping configurations under the specified project.​
ProjectName​
String​
No​
fa************​
Specify the project name. Used to query log shipping configuration under the specified project. Support fuzzy match.​
IamProjectName​
String​
No​
default​
Specify the IAM project name. Used to query log shipping configurations under the specified IAM project.​
PageNumber​
Integer​
No​
10​
Page size. Default is 20, maximum is 100.​
PageSize​
Integer​
No​
20​
Page number when performing a paginated query. By default, this is 1, which means data is returned starting from the first page.​
ShipperName​
String​
No​
test​
Specify the log shipping configuration name; supports fuzzy matching.​
ShipperId​
String​
No​
f3e901c3-b17f-42fd-aa8c-dc91a6c7****​
Specify the log shipping configuration ID.​
TopicName​
String​
No​
test​
Specify the log topic name. Used to query the log shipping configuration for this log topic. Fuzzy matching supported.​
TopicId​
String​
No​
f3e901c3-b17f-42fd-aa8c-dc91a6c7****​
Specify the log topic ID. Used to query log shipping configurations related to this log topic.​
ShipperType​
String​
No​
tos​
Specify the log shipping type to query its related log shipping configuration. Available options include:​
  • tos: ship to TOS.​
  • kafka: ship to Kafka.​
​
Response parameters​
The table below lists only the unique response parameters for this API. For more information, see Response Structure.​
​
Parameter​
Type​
Example value​
Description​
Shippers​
Array of DescribeResp​
/​
Information related to delivery configuration.​
Total​
Integer​
5​
Number of log shipping configurations.​
​
DescribeResp​
​
Parameter​
Type​
Example value​
Description​
Status​
Boolean​
true​
Enable shipping configuration.​
TopicId​
String​
c7****​
ID of the log topic containing the log to be shipped.​
ProjectId​
String​
dc7****​
ID of the project where the log to be shipped is located.​
ShipperId​
String​
4a9bd4bd-53f1-43ff-b88a-64ee1be5****​
Log shipping configuration ID.​
TopicName​
String​
mytopic​
Name of the log topic where the logs to be shipped are stored.​
CreateTime​
String​
2024-08-01 14:01:02​
Creation time of the log shipping configuration.​
ModifyTime​
String​
2024-08-01 14:01:10​
Last modified time of the shipping configuration.​
ContentInfo​
Object of ContentInfo​
/​
Content format configuration for log delivery.​
DashboardId​
String​
/​
The default built-in dashboard ID for shipping.​
ProjectName​
String​
myproject​
Name of the project where the logs to be shipped are located.​
ShipperName​
String​
tosship​
Delivery configuration name.​
ShipperType​
String​
tos​
Shipping type.​
ShipperEndTime​
Integer​
1723709712496​
Log shipping end time.​
TosShipperInfo​
Object of TosShipperInfo​
/​
Information delivered to TOS​
KafkaShipperInfo​
Object of KafkaShipperInfo​
/​
Information delivered to Kafka.​
ShipperStartTime​
Integer​
1723709602496​
Log shipping start time.​
RoleTrn​
String​
trn:iam::2100000001:role/TLSETLAccessForUserA​
Trn for custom roles.​
​
ContentInfo​
​
Parameter​
Type​
Example value​
Description​
Format​
String​
json​
Log content parsing format.​
  • Supports json, jsonl and csv configuration for delivery to TOS.​
  • Supports original and json configuration for delivery to Kafka.​
CsvInfo​
Object of CsvInfo​
/​
CSV format log content configuration.​
JsonInfo​
Object of JsonInfo​
/​
Log content configuration in JSON/JSONL format.​
​
TosShipperInfo​
​
Parameter​
Type​
Example value​
Description​
Bucket​
String​
mytos​
Name of the object storage bucket.​
Prefix​
String​
testprefix​
Name of the top-level directory in the bucket. All log data shipped by this configuration will be stored under this directory.​
MaxSize​
Integer​
5​
Maximum raw file size that can be shipped per shard (Shard), which refers to the uncompressed log file size Measured in MiB​
Compress​
String​
snappy​
Compression format.​
Interval​
Integer​
300​
Log shipping time interval (in seconds).​
PartitionFormat​
String​
%Y/%m/%d/%H/%M​
Shard rules for log shipping logs.​
​
KafkaShipperInfo​
​
Parameter​
Type​
Example value​
Description​
EndTime​
Integer​
1723709712496​
End time for log shipping, in milliseconds. If not set, shipping continues indefinitely​
tip​
When shipping to Kafka, it is recommended to set the shipping time range using the ShipperStartTime and ShipperEndTime fields, and leave the StartTime and EndTime fields blank​
Compress​
String​
snappy​
Compression format; currently supports snappy, gzip, lz4, and none.​
Instance​
String​
kafka-cnngbnntswg1****​
Kafka instance.​
StartTime​
Integer​
1723709602496​
Delivery start time in milliseconds. If not specified, the default is the current time.​
tip​
When delivering to Kafka, use the ShipperStartTime and ShipperEndTime fields to set the delivery time range, and leave the StartTime and EndTime fields empty.​
KafkaTopic​
String​
topic-c​
Kafka Topic name. All log data delivered by this configuration will be sent to this Topic.​
​
CsvInfo​
​
Parameter​
Type​
Example value​
Description​
Keys​
Array of String​
["user","log","ip"]​
Configure the fields that need to be shipped.​
  • Only letters, numbers, and _-./ are supported._Cannot begin with.​
  • Within the same index, the key name must be unique.​
  • Length must be between 1 and 128 characters.​
Delimiter​
String​
,​
Separator: Supports comma, tab, vertical bar, semicolon, and space.​
EscapeChar​
String​
false​
Escape character: When the field contains a separator, use an escape character to wrap the field. Currently, only single quotes, double quotes, and empty characters are supported.​
PrintHeader​
Boolean​
false​
Print the Key in the first row.​
NonFieldContent​
String​
test​
Content for invalid field fill, length 0–128​
​
JsonInfo​
​
Parameter​
Type​
Example value​
Description​
Keys​
Array of String​
["__content", "__pod_name__"]​
When using JSON/JSONL format for shipping, if this parameter is not configured, all fields will be shipped, including __content__ (required), __source__, __path__, __time__, __image_name__, __container_name__, __pod_name__, __pod_uid__, namespace, __tag____client_ip__, and __tag____receive_time__.​
Enable​
Boolean​
true​
Enable flag.​
Escape​
Boolean​
true​
Escape characters supported. This must be set to true.​
​
Sample request​
​
GET https://{tls-endpoint}/DescribeShippers?&ProjectId=f3e901c3-b17f-42fd-aa8c-dc91a6c7****&ProjectName=&IamProjectName=&PageNumber=10&PageSize=20&ShipperName=test&ShipperId=f3e901c3-b17f-42fd-aa8c-dc91a6c7****&TopicName=test&TopicId=f3e901c3-b17f-42fd-aa8c-dc91a6c7****&ShipperType=tos HTTP/1.1​
Content-Type: application/json​
​
Sample response​
​
{​
"Shippers": [​
{​
"ShipperId": "62f555d9-d****8c8-f47cf7ab2f8e",​
"ShipperName": "csv-search-log",​
"ProjectId": "d11be*****47cf-8170-72c318262132",​
"ProjectName": "log-sy",​
"TopicId": "eca85523******56-b750-681141d6aa8a",​
"TopicName": "search-log",​
"Status": true,​
"ShipperType": "tos",​
"TosShipperInfo": {​
"Bucket": "tls-topic-shipper",​
"Prefix": "csv-search-log",​
"PartitionFormat": "%Y/%m/%d/%H",​
"MaxSize": 256,​
"Interval": 300,​
"Compress": "snappy"​
},​
"ContentInfo": {​
"Format": "csv",​
"CsvInfo": {​
"PrintHeader": true,​
"Keys": [​
"account_id",​
"status",​
"Action",​
"ReqID",​
"__image_name__",​
"__content__",​
"__time__"​
],​
"Delimiter": ",",​
"EscapeChar": "'",​
"NonFieldContent": ""​
},​
"JsonInfo": null​
},​
"ShipperStartTime": 1725012764000,​
"ShipperEndTime": null,​
"CreateTime": "2024-08-22 12:23:16",​
"ModifyTime": "2024-08-30 18:12:44"​
},​
{​
"ShipperId": "6e948fca-d3*****-b5cca0b29383",​
"ShipperName": "tosTimeTest",​
"ProjectId": "d11be542-3ba2******70-72c318262132",​
"ProjectName": "log-sy",​
"TopicId": "680284a6-8e88-4d*****1-d416a48c45ad",​
"TopicName": "import-task",​
"Status": true,​
"ShipperType": "tos",​
"TosShipperInfo": {​
"Bucket": "tos-import-logs",​
"Prefix": "timeTest",​
"PartitionFormat": "%Y",​
"MaxSize": 256,​
"Interval": 60,​
"Compress": "snappy"​
},​
"ContentInfo": {​
"Format": "json",​
"CsvInfo": null,​
"JsonInfo": {​
"Enable": true,​
"Escape": false,​
"Keys": null​
}​
},​
"ShipperStartTime": 1725012764000,​
"ShipperEndTime": 1723795560000,​
"CreateTime": "2024-08-16 16:06:39",​
"ModifyTime": "2024-08-30 18:12:44"​
}​
],​
"Total": 2​
}​
​
Error Code​
You can visit Public Error Code to get more information about error codes.​
Last updated: 2026.02.03 17:43:46