You need to enable JavaScript to run this app.
Torch Log Service

Torch Log Service

Copy page
Download PDF
Data transformation API
DescribeETLTasks
Copy page
Download PDF
DescribeETLTasks
Call the DescribeETLTasks API to retrieve the list of data transformation tasks.​
Request instructions​
  • Request method: GET​
  • Request URL: https://{tls-endpoint}/DescribeETLTasks​
Request parameters​
The table below lists only request-specific parameters for this API and some common parameters. For more information, see common parameters.​
Query​
​
Parameter​
Type​
Required​
Example value​
Description​
ProjectId​
String​
No​
f3e901c3-b17f-42fd-aa8c-dc91a6c7****​
Specify the project ID to query data transformation tasks under the selected project.​
ProjectName​
String​
No​
myproject​
Specify the project name to query data transformation tasks under the specified project. Supports fuzzy search.​
IamProjectName​
String​
No​
default​
Specify the IAM project name to query data transformation tasks in the designated IAM project.​
SourceTopicId​
String​
No​
f3e901c3-b17f-42fd-aa8c-dc91a6c7****​
Specify the log topic ID to query data transformation tasks related to this topic​
SourceTopicName​
String​
No​
topic​
Specify the log topic name to query related data transformation tasks for that topic Supports fuzzy matching​
TaskId​
String​
No​
f3e901c3-b17f-42fd-aa8c-dc91a6c7****​
Processing task ID.​
TaskName​
String​
No​
test-task​
Transformation task name.​
Status​
String​
No​
WAITING​
Specify the data transformation task status to query tasks in that state.​
PageNumber​
Integer​
No​
10​
Page number for paginated queries. Default is 1, which returns data starting from the first page.​
PageSize​
Integer​
No​
20​
Page size. The default is 20 and the maximum is 100.​
​
Response parameters​
The table below lists only the response parameters specific to this API. For more information, see response structure.​
​
Parameter​
Type​
Example value​
Description​
Tasks​
/​
Transformation task details​
Total​
Integer​
5​
Number of processing tasks.​
​
DescribeETLTaskResp​
​
Parameter​
Type​
Example value​
Description​
Name​
String​
etl-task-name​
Transformation task name​
Enable​
Boolean​
true​
Transformation task status​
Script​
String​
f_set("key","value")​
Processing rules.​
TaskId​
String​
4a9bd4bd-53f1-43ff-b88a-64ee1be5****​
Processing task ID.​
ToTime​
Integer​
1643684482​
End time for log processing; leave blank for no limit.​
DSLType​
String​
NORMAL​
DSL type, fixed as NORMAL.​
FromTime​
Integer​
1643684482​
Start time for log processing. Empty means all historical data​
TaskType​
String​
Resident​
Task type, fixed as Resident​
ETLStatus​
String​
WAITING​
Task scheduling status​
  • Waiting: preparing.​
  • Running: in progress.​
  • Stopping: stopping.​
  • Stopped: stopped.​
  • Restarting: restarting.​
ProjectId​
String​
f***********​
ID of the project containing logs to be processed.​
CreateTime​
String​
2022-12-01 14:01:02​
Processing task creation time.​
ModifyTime​
String​
2022-12-01 14:01:02​
Last modified time of the processing task​
Description​
String​
This is a test project​
Brief description of the transformation task.​
ProjectName​
String​
myproject​
Name of the project where data to be transformed is located.​
SourceTopicId​
String​
4a9bd4bd-53f1-43ff-b88a-64ee1be5****​
ID of the log topic containing logs to be transformed​
LastEnableTime​
String​
2022-12-01 14:01:02​
Recent startup time.​
SourceTopicName​
String​
test-topic​
Name of the log topic containing logs to be processed​
TargetResources​
targettopic​
Log topic for the target output.​
​
TargetResourcesResp​
​
Parameter​
Type​
Example value​
Description​
Alias​
String​
Custom alias​
Custom name for the output target.​
TopicId​
String​
4a9bd4bd-53f1-43ff-b88a-64ee1be5****​
ID of the log topic used to store transformed logs​
ProjectId​
String​
4a9bd4bd-53f1-43ff-b88a-64ee1be5****​
Project ID for storing logs after data transformation.​
TopicName​
String​
topicname​
Name of the log topic for storing logs after data transformation.​
ProjectName​
String​
projectname​
Name of the project used to store transformed log entries​
Region​
String​
cn-beijing​
The region where the project that stores transformed logs is located.​
RoleTrn​
String​
trn:iam::2100000001:role/TLSETLAccessForUserA​
Name of the cross-account authorization role You can log in to the IAM console, click the corresponding role name under Role Management, and view the Role TRN in Basic Information​
​
Sample request​
​
GET https://{tls-endpoint}/DescribeETLTasks?&ProjectId=f3e901c3-b17f-42fd-aa8c-dc91a6c7****&ProjectName=&IamProjectName=&SourceTopicId=f3e901c3-b17f-42fd-aa8c-dc91a6c7****&SourceTopicName=topic名称&TaskId=f3e901c3-b17f-42fd-aa8c-dc91a6c7****&TaskName=test-task&Status=WAITING&PageNumber=10&PageSize=20 HTTP/1.1​
Content-Type: application/json​
​
Response example​
​
{​
"Total": 2,​
"Tasks": [​
{​
"TaskId": "e9b00e4a-ae***c4-83e5-a38d7f73ebfc",​
"Name": "zy-access-",​
"SourceTopicId": "5687efb0-4ada*****5-4447770a3fe6",​
"SourceTopicName": "access-log",​
"DSLType": "NORMAL",​
"Script": "e_keep(f_has(\"__time__\"))",​
"TaskType": "Resident",​
"TargetResources": [​
{​
"TopicId": "e2cc153d-3ba8-4****80e1ef1bead9",​
"TopicName": "topic1",​
"ProjectId": "4012777e-c******dc4b3a4ab",​
"ProjectName": "zy-test",​
"Alias": "target",​
"Region": "cn-beijing"​
}​
],​
"Enable": true,​
"Description": "This is a test project",​
"ETLStatus": "RUNNING",​
"FromTime": 1709025132,​
"CreateTime": "2024-02-27 17:12:30",​
"ModifyTime": "2024-02-27 17:12:49",​
"LastEnableTime": "2024-02-27 17:12:30",​
"ProjectId": "d11be542-****170-72c318262132",​
"ProjectName": "log-sy"​
},​
{​
"TaskId": "597887ea-b6b****8289b92fec2",​
"Name": "search-server-test",​
"SourceTopicId": "ffbcf0dd****9bc2-e5702e159e23",​
"SourceTopicName": "search-server-log",​
"DSLType": "NORMAL",​
"Script": "e_keep(f_search(\"__content__:\\\"task done\\\"\"))\next_kv(\"__content__\", pair_sep=\"] [\", kv_sep=\":\", prefix=\"\", suffix=\"\", mode=\"fill\")\nf_set(\"ReqId\", regex_replace(v(\"__content__\"), \".*ReqId:([0-9a-z_.:]*).*\", replace=\"$1\"))\nf_drop(\"__content__\", \"__context_flow__\", \"\")\nf_set(\"insertResultCost\", str_replace(v(\"insertResultCost\"), \"us]\", \"\", 1))\nf_set(\"searchTotalCost\", str_replace(v(\"searchTotalCost\"), \"us]\", \"\", 1))\nf_set(\"resultSize\", str_replace(v(\"resultSize\"), \"]\", \"\", 1))\nf_set(\"ReadRawQuotaTotalCostUs\", str_replace(v(\"ReadRawQuotaTotalCostUs\"), \"]\", \"\", 1))\ne_keep(f_has(\"HitFiles\"))",​
"TaskType": "Resident",​
"TargetResources": [​
{​
"TopicId": "affa167b-d2****ec-e202a98cfa7a",​
"TopicName": "search-server-static",​
"ProjectId": "d11be542-3ba2*****72c318262132",​
"ProjectName": "log-sy",​
"Alias": "target",​
"Region": "cn-beijing"​
}​
],​
"Enable": true,​
"Description": "",​
"ETLStatus": "RUNNING",​
"FromTime": 1709010000,​
"CreateTime": "2024-02-27 17:08:10",​
"ModifyTime": "2024-02-27 17:08:30",​
"LastEnableTime": "2024-02-27 17:08:10",​
"ProjectId": "d11be542-3ba2*****318262132",​
"ProjectName": "log-sy"​
}​
]​
}​
​
Error Code​
You can visit Public Error Code for more information about error codes.​
Last updated: 2025.10.28 18:42:02