Docs

Intelligent Data Insight

Dataset Open API

Dataset Information API

1. Overview

This document introduces the "Open API for dataset information," including how users can use the API to view basic information on the dataset details and edit pages, update basic information, set tags, and more.
Caution: Before using the APIs described in this document, you must complete the prerequisites of integrating JWT-Token and applying for a Token. Please first understand the invocation method. For details, see Invocation Method.

2. API introduction

2.1 View dataset information - details page

Permission requirements
Resource
Permission
Dataset
read
API description and notes: Retrieve the dataset information displayed on the dataset details page.

Resource	Permission
Dataset	read

Request description

Request URL: {{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/overview

Request example

curl --location --request GET '{{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/overview' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {{jwtToken}}'

Request parameters

Name	Type	Required	Example value	Description
dataSetId	integer	Yes	123	For how to obtain it, see How to Obtain Resource IDs

Response parameters

Parameter	Type	Description
accessCount	integer	Page views (PV) of related chart pages for the dataset in the past 30 days
appId	integer	Project ID
connectionMode	integer	Dataset connection mode [0: extraction, 1: direct connection]
ctime	datetime	Dataset creation time
dagTagConf.dimTbNodes	list	Dimension table node ID
demoUrl	string	Dataset usage instructions
descr	string	Dataset description
doradoPriority	string	Synchronization task priority [normal, high, super_high]
driverName	string	Database driver name
editable	boolean	Whether the dataset can be modified [true, false]
frequency	string	Dataset synchronization frequency [daily, hourly, weekly, monthly]
hot	integer	Access popularity [0: no popularity]
id	integer	Dataset ID
lastSyncTime	datetime	Last synchronization time
lineNum	integer	Number of data entries stored in the dataset
mtime	datetime	Last modification time of the dataset
name	string	Dataset name
onPrivateCluster	integer	Whether the dataset is created on a private cluster [0: not a private cluster; 1: private cluster]
roleList	list	Dataset permissions held by the current token identity [admin owner, read_role read role, write_role write role, view_only_role view-only role, read_all_role global view role]
size	float	Data size stored in the dataset [unit is dynamic, may be KB, MB, GB...]
sourceKey	string	Dataset creation/output source [abi, cdp, prep] Note: prep indicates a dataset generated through visual modeling
status	integer	Dataset status; 0 indicates normal status
syncMode	integer	Synchronization mode [0: Bdb various to ClickHouse channel tasks, 1: visual modeling tasks, 2: Bdp hsql tasks, using tqs for import]
syncType	integer	Synchronization scheduling mode [1: automatic synchronization; 2: manual synchronization]
ttl	integer	Dataset lifecycle -> Data retention period, default is 7 days Note: The definition is the dataset lifecycle, but it actually refers to the data retention period of the dataset. For example, if the default is 7 days, all data generated more than 7 days ago will be deleted.
type	integer	Dataset type [Common types: 0: Dataset generated from a ClickHouse table as the data source, 2: Dataset generated from a Hive table as the data source, 8: Dataset generated from custom SQL, 10: Dataset generated from a MySQL table as the data source, 22: Prep type dataset]
writePartition	integer	Write partition [0 for day partition; 1 for day + hour partition]
sensitiveConf	string	Dataset data masking settings [undef: not set; sensitiveWithoutDe: sensitive but not masked; sensitiveWithDe: sensitive and masked; unsensitive: not sensitive]

Response example

{
    "code": "aeolus/ok",
    "data": {
        "accessCount": 0,
        "appId": 77*****,
        "connectionMode": 0,
        "ctime": "2022-12-14 12:17:54",
        "dagTagConf": {
            "dimTbNodes": []
        },
        "dataSetLabelType": 0,
        "demoUrl": "http://xxx.com",
        "descr": "这是一段描述，描述数据集的作用",
        "disablePreview": 0,
        "doradoPriority": "normal",
        "driverName": "click_house",
        "editable": true,
        "frequency": "daily",
        "groupType": 0,
        "hot": 0,
        "id": {{dataSetId}},
        "isAppAdminView": 1,
        "lastSyncTime": "2022-12-14 12:38",
        "lineNum": "56",
        "mtime": "2022-12-14 12:18",
        "name": "测试open-api专用数据集1",
        "onPrivateCluster": 0,
        "ownerEmailPrefix": "userEmailPrefix",
        "parentId": 3075,
        "partitionRange": [
            {
                "id": 158**********,
                "max": "2022-12-13",
                "min": "2022-12-13",
                "name": "p_date"
            }
        ],
        "previewInfo": "",
        "retryNum": 0,
        "roleList": [
            {
                "action": "admin",
                "roleName": "所有者"
            },
            {
                "action": "view_only_role",
                "roleName": "仅阅览角色"
            },
            {
                "action": "write_role",
                "roleName": "写角色"
            },
            {
                "action": "read_role",
                "roleName": "读角色"
            }
        ],
        "size": "2.86KB",
        "slaStatus": 0,
        "sourceKey": "abi",
        "status": 0,
        "statusDesc": "正常",
        "syncMode": 2,
        "syncType": 1,
        "tagInfo": null,
        "thirdPartyConf": {},
        "ttl": 7,
        "type": 22,
        "uneditableReason": null,
        "writePartition": 0,
        "yarnName": "root.default_hrtech_da_pm"
    },
    "msg": "成功"
}

2.2 View complete dataset information - edit page

Permission requirements
Resource
Permission
Dataset
read
API description and notes: Retrieve the complete dataset information, which corresponds to the dataset edit page on the frontend.

Resource	Permission
Dataset	read

Request description

Request URL: {{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/info

Request example

curl --location --request GET '{{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/info' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {{jwtToken}}'

Request parameters

Parameter	Type	Required	Example value	Description
dataSetId	integer	Yes	123	Dataset ID, see How to Obtain Resource IDs

Response parameters

Parameter (jsonPath)	Type	Description
$.baseConf	object	2.2 View complete dataset information - edit page
$.baseConf.appId	integer	Project ID
$.baseConf.demoUrl	string	Dataset usage instructions
$.baseConf.name	string	Dataset name
$.baseConf.ownerEmailPrefix	string	Dataset owner
$.baseConf.parentId	integer	Folder containing the dataset; 0 indicates the root directory
$.baseConf.syncMode	integer	Synchronization mode [1 for manual; 2 for automatic]
$.baseConf.type	integer	Dataset type [Common types: 0: Dataset generated from a ClickHouse table as the data source, 2: Dataset generated from a Hive table as the data source, 8: Dataset generated from custom SQL, 10: Dataset generated from a MySQL table as the data source, 22: Prep type dataset]
$.dagTagConf.dimTbNodes	list	Dataset dimension table node nodeId
$.syncConf	object	Dataset Synchronization Task API
$.whereConf	object	Dataset model data source node filter conditions and global filter conditions

Response example

{
    "code": "aeolus/ok",
    "data": {
        "baseConf": {
            "appId": 77*****,
            "connectionMode": 0,
            "demoUrl": null,
            "descr": "这是一段描述，描述数据集的作用",
            "generalTags": [],
            "groupType": 0,
            "id": {{dataSetId}},
            "isAuthEnabled": 1,
            "isIntelligentSyncEnable": 1,
            "lineOfBusiness": [],
            "name": "测试open-api专用数据集1",
            "ownerEmailPrefix": "userEmailPrefix",
            "parentId": 3075,
            "productName": "",
            "regionType": [],
            "syncMode": 2,
            "thirdPartyConf": null,
            "type": 22
        },
        "dagTagConf": {
            "dimTbNodes": []
        },
        "dimMetList": [
            {
                "calcType": null,
                "dataFormat": null,
                "dataTypeName": "date",
                "defaultDataTypeName": "date",
                "descr": "p_date",
                "dimMetCategoryId": null,
                "dimMetCategoryType": null,
                "dimMetMixOrder": 0,
                "dimMetOrder": 0,
                "dimMetThirdPartyConf": null,
                "dimMetVariety": 1,
                "editable": 0,
                "errCode": "ok",
                "expr": "p_date",
                "exprAsDim": null,
                "fieldList": "[\"p_date\"]",
                "fullExpr": "p_date",
                "groupType": 0,
                "hitFilterRuleList": [],
                "id": 158**********,
                "isDeletedField": 0,
                "isPartitionField": 1,
                "lodType": null,
                "mapType": 0,
                "name": "p_date",
                "operationCode": 0,
                "ownerEmailPrefix": "userEmailPrefix",
                "paramIdList": [],
                "showExpr": 1,
                "visible": 1
            },
            .........
        ],
        "labelConf": {
            "dataSetLabelType": 0,
            "labelParams": null,
            "subjectList": null
        },
        "linkConf": [],
        "nodeConf": [
            {
                "clusterName": "cn",
                "dataSourceId": 25***,
                "dataSourceType": "ch_excel",
                "dbName": "aeolus_excel_upload_aeolus_toy",
                "displayDbName": "配比四刷-v4",
                "fields": [
                    {
                        "alias": "`字段四`",
                        "comment": null,
                        "isDynamicPartition": false,
                        "isSelect": true,
                        "isSourceTableField": false,
                        "name": "字段四",
                        "prepType": "string",
                        "status": 0,
                        "type": "Nullable(String)"
                    },
                    {
                        "alias": "`字段一`",
                        "comment": null,
                        "isDynamicPartition": false,
                        "isSelect": true,
                        "isSourceTableField": false,
                        "name": "字段一",
                        "prepType": "string",
                        "status": 0,
                        "type": "Nullable(String)"
                    },
                    {
                        "alias": "`字段三`",
                        "comment": null,
                        "isDynamicPartition": false,
                        "isSelect": true,
                        "isSourceTableField": false,
                        "name": "字段三",
                        "prepType": "string",
                        "status": 0,
                        "type": "Nullable(String)"
                    },
                    {
                        "alias": "`字段二`",
                        "comment": null,
                        "isDynamicPartition": false,
                        "isSelect": true,
                        "isSourceTableField": false,
                        "name": "字段二",
                        "prepType": "string",
                        "status": 0,
                        "type": "Nullable(String)"
                    }
                ],
                "fileId": "13516",
                "fullOption": false,
                "hiveOwner": "userEmailPrefix",
                "lastDdlTime": "2022-12-13 12:49:00",
                "metadataLink": "https://data.bytedanc*.net/coral/datamap/detail/table_info/clickhouse/aeolus_excel_upload_aeolus_toy/table_4f0cc551_9dfa_****_****_**********#group=default",
                "nodeId": "77b3df39-834c-4da4-908e-4b54846bc599",
                "nodeType": "table",
                "params": {
                    "extractType": "{\"key\":\"full\"}"
                },
                "partitionConfList": [],
                "query": "",
                "schemaName": "Sheet1",
                "tableAlias": "table_4f0cc551_9dfa_4915_****_************",
                "tableRowFilter": {},
                "tbId": "cn//aeolus_excel_upload_aeolus_toy//table_4f0cc551_9dfa_4915_****_************",
                "tbName": "table_4f0cc551_9dfa_4915_****_************"
            }
        ],
        "parseEngine": 0,
        "syncConf": {
            "backtrackingConf": {
                "dateRange": {
                    "endDate": "2022-12-13",
                    "startDate": "2022-12-13"
                },
                "enable": 1
            },
            "doradoPriority": "normal",
            "frequency": "daily",
            "monitorConf": {
                "alarmRules": [
                    {
                        "failedAlarmItems": [
                            {
                                "item": "retry_failed"
                            }
                        ],
                        "id": 6826,
                        "name": "auto_aeolus_rule_{{dataSetId}}_167099*******_dev",
                        "normalNoticeConf": [
                            {
                                "dutySchedules": [],
                                "noticeChannel": "lark",
                                "users": [
                                    "userEmailPrefix"
                                ]
                            }
                        ],
                        "resultAlarmItems": [],
                        "timeoutAlarmItems": []
                    }
                ]
            },
            "paramsConfList": [
                {
                    "name": "tqs.query.auto.retry.enable",
                    "value": "true"
                }
            ],
            "performanceSettings": {
                "chQueryParams": {},
                "clusterName": "cnch_alpha_lq",
                "cnchVwDefault": "vw_default",
                "cnchVwWrite": "vw_write",
                "dataSourceId": 14*,
                "driverName": "click_house",
                "partitionKey": [
                    "p_date"
                ],
                "primaryKeyList": [],
                "sampleKey": null,
                "sampleRate": 1.0,
                "shardKey": null
            },
            "retryInterval": 5,
            "retryNum": 1,
            "scheduleDay": "0",
            "scheduleTime": "00:00",
            "scheduleTimeType": 0,
            "slaDaytime": null,
            "slaStatus": 0,
            "syncType": 1,
            "ttl": 7,
            "upstreamSettings": {
                "dynamicPartitionConfList": [],
                "dynamicPartitionMode": false,
                "partitionConfList": [],
                "uniqueIndexList": []
            },
            "writePartition": 0,
            "yarnName": "root.default_hrtech_da_pm"
        },
        "whereConf": {
            "nodeRowFilter": {},
            "requiredRowFilter": []
        }
    },
    "msg": "成功"
}

2.3 View basic dataset information

Permission requirements
Resource
Permission
Dataset
read
API description and notes: Retrieve the basic dataset information, which can be used for a summary display of the dataset content for third-party integrations.

Resource	Permission
Dataset	read

Request description

Request URL: {{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/digest

Request example

curl --location --request GET '{{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/digest' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {{jwtToken}}'

Request parameters

Name	Type	Required	Example value	Description
dataSetId	integer	Yes	123	Dataset ID, see How to Obtain Resource IDs
folderPath	integer	No	0	Whether the complete folder path is required [0 not required; 1 required]

Response parameters

Name	Type	Description
id	integer	Dataset ID
appID	integer	Project ID
appName	string	Project name
detailUrl	string	Dataset frontend page address, can be directly redirected
folderName	list	Dataset folder path -> The root directory is the default value, indicating that this dataset is not stored in a folder If the input parameter folderPath=0, then in the following JSON example, folderName is only "folderName": [ "Subfolder - Level 3" ]
name	string	Dataset name
ownerEmailPrefix	string	Dataset owner
status	integer	Dataset status -> Normally, status=0
type	integer	Dataset types [Common types: 0: Dataset generated from a ClickHouse table as the data source, 2: Dataset generated from a Hive table as the data source, 8: Dataset generated from custom SQL, 10: Dataset generated from a MySQL table as the data source, 22: Prep-type dataset]

{
    "code": "aeolus/ok",
    "data": {
        "appId": 7777947,
        "appName": "测试项目-名称",
        "demoUrl": "http://xxx.com",
        "descr": "这是一段描述，描述数据集的作用",
        "detailUrl": "https://aeolus-release.fedev.bytedanc*.net/aeolus#/dataManage/detail/{{dataSetId}}?appId=77*****",
        "folderName": [
            "子文件夹-层级3",
            "子文件夹-层级2",
            "子文件夹-层级1",
            "模板数据集",
            "根目录"
        ],
        "id": {{dataSetId}},
        "name": "测试open-api专用数据集1",
        "ownerEmailPrefix": "userEmailPrefix",
        "status": 0,
        "type": 22
    },
    "msg": "成功"
}

2.4 Update basic dataset information

Permission requirements
Resource
Permission
Dataset
write
API description and notes: Corresponds to updating the basic description information of the dataset on the frontend.

Resource	Permission
Dataset	write

Caution: The information that can be updated by this API differs from that of the View basic dataset information API.

Request description

Request URL: {{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/digest
Caution: This is a full overwrite update. If you need to update a certain property, you must also provide the original content of all other properties in full. Otherwise, the existing properties will be overwritten with blank values.

Request example

curl --location --request PUT '{{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/digest' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {{jwtToken}}' \
--data-raw '{
    "dataSetName": "测试open-api专用数据集1",
    "dataSetDescr": "这是一段描述，描述数据集的作用"
}'

Request parameters

Parameter	Type	Required	Example value	Description
dataSetId	integer	Yes	123	Dataset ID. For how to obtain, see How to Obtain Resource IDs
dataSetName	integer	Yes	Dataset 1	Dataset name
dataSetDescr	string	Yes	This is a dataset description	Dataset description

Response example

{
    "code": "aeolus/ok",
    "data": {
        "affectedRows": 1
    },
    "msg": "成功"
}

2.5 Set dataset tags

Permission requirements
Resource
Permission
Dataset
write
Interface description and notes: Set the tag information for a dataset. Dataset tags are associated with the project and must be used together with the "Dataset tag - Retrieve dataset tag information under a project" API.

Resource	Permission
Dataset	write

Request description

Request URL: {{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/dataSetTag

Request example

curl --location --request POST '{{domain}}/aeolus/api/v4/open/dataset/{{dataSetId}}/dataSetTag' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {{jwtToken}}' \
--data-raw '{
    "tagIdList": [
        153
    ]
}'

Request parameters

Name	Type	Required	Example value	Description
dataSetId	integer	Yes	123	Dataset ID. For how to obtain, see How to Obtain Resource IDs
tagIdList	list < integer >	Yes	153	Tag ID. Multiple tags can be set for a dataset at once; the frontend allows selecting multiple tags

Response example

{
    "code": "aeolus/ok",
    "data": {},
    "msg": "成功"
}

2.6 Retrieve dataset folder list under a project

Version: V2.76.0 and above

Permission requirements

Resource	Permission
Project	read

Request description

Retrieve folder list under a project

API path
{{domain}}/aeolus/api/v4/open/app/{{appId}}/dataSetFolderTreeV2

Request example

cURL example

curl --location --request GET '{{domain}}/aeolus/api/v4/open/app/{{appId}}/dataSetFolderTreeV2' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {{jwtToken}}'

Request parameters

Parameter	Type	Required	Example value	Description
appId	integer	Yes	123	Project ID. For how to obtain, see How to Obtain Resource IDs

Response parameters

Name	Type	Description
official	list	Official directory
private	list	Personal directory
public	list	Public directory
appID	integer	Project ID
id	integer	Dataset ID
belong	integer	1 is a public dataset, 2 is a personal dataset
ctime	datetime	Dataset creation time
deleteTime	datetime	Dataset deletion time
mtime	datetime	Dataset last modified time
descr	string	Dataset description
name	string	Dataset name
ownerEmailPrefix	string	Dataset owner
status	integer	Dataset folder status: 0 is normal, 1 is permanently deleted, 2 is in the recycle bin
parentId	integer	Parent folder of the dataset folder. If 0, it means there is no parent folder and it is already in the root directory
childList	list	List of subfolders in the dataset folder

Response example

{
    "code": "aeolus/ok",
    "data": {
        "official": [
            {
              "appId": 7*******,  #项目id
              "belong": 1,     #归属 1是公共，2是个人
              "ctime": "2024-05-30 11:57",
              "deleteTime": null,
              "descr": null,
              "id": 12107,
              "mtime": "2024-05-30 11:57",
              "name": "测试",
              "ownerEmailPrefix": "userEmailPrefix",
              "parentId": 12101,  #父目录id
              "status": 0,   
              "childList": [],   #子目录list
            }
        ],
        "private": [],
        "public": [],
    }
    "msg": "成功"
}

Last updated: 2026.07.06 17:03:29

Intelligent Data Insight

2.1 View dataset information - details page #

Request description

Request example

Request parameters

Response parameters

Response example

2.2 View complete dataset information - edit page #

Request description

Request example

Request parameters

Response parameters

Response example

2.3 View basic dataset information #

Request description

Request example

Request parameters

Response parameters

2.4 Update basic dataset information #

Request description

Request example

Request parameters

Response example

2.5 Set dataset tags #

Request description

Request example

Request parameters

Response example

2.6 Retrieve dataset folder list under a project #

Request description

Request example

Request parameters

Response parameters

Response example

2.1 View dataset information - details page

2.2 View complete dataset information - edit page

2.3 View basic dataset information

2.4 Update basic dataset information

2.5 Set dataset tags

2.6 Retrieve dataset folder list under a project