1.1 Overview #

Data lineage analysis is a method used to trace and understand the origin, flow, and evolution of data throughout an entire system or process, helping users manage data more efficiently.
With data lineage analysis, users can query the full-chain upstream and downstream dependencies of resources such as dashboards, charts, datasets, visual modeling, large screens, data sources/data connections, and more, gaining a clear understanding of how data is generated, the processing steps and transformations it undergoes, and which resources ultimately use it. When troubleshooting issues, data lineage analysis also helps users quickly locate the source of problems, such as tracing upstream to identify key nodes that introduce metric calculation issues, or evaluating downstream the impact scope of metric definition changes. The data lineage analysis feature is crucial for ensuring data reliability, availability, and value realization, as well as for improving an enterprise's data management and business decision-making capabilities.

1.2 Application scenarios #

• Master the data analysis chain: Through resource lineage, you can see the relationships among each data source, dataset, visual modeling task, chart, dashboard, digital large screen, and more, making it easy to understand the data analysis process.
• Assess resource deletion and modification risks: Understanding upstream and downstream dependencies of resources enables a more comprehensive assessment of the risks associated with deleting or modifying resources. When troubleshooting, you can also quickly locate the source of problems, such as tracing upstream to identify key nodes that introduce metric calculation issues, or evaluating downstream the impact scope of metric definition changes.

2.1 Accessing data lineage analysis #

(1) Click Project Center.
(2) Click Data Lineage Analysis on the left to enter the data lineage analysis interface, select resource type, object, analysis granularity, and view the full-chain relationships among resources and more.

2.2 Querying resource lineage #

As shown in the figure, users can query by resource type, then resource object, and then analysis granularity to view the end-to-end lineage analysis of a resource.
(1) Select the resource type. Users can query resources of the following types: dashboard, chart, dataset, visual modeling, large screen, and data source/data connection:

(2) Under the selected resource type, select the resource object for which you want to view the end-to-end lineage analysis.

(3) Select the analysis granularity. If the resource type is chart, dataset, or data source/data connection, you can select the analysis granularity to view lineage analysis at different granularities:

For example, for the dataset resource type, if you select dimension/metric as the analysis granularity, you can select a field in the upper right corner to perform end-to-end lineage analysis for a single field.

2.3 Understand lineage details #

(1) After selecting the resource for which you want to view lineage, the system will automatically count the number of associated resources, as well as the types and counts of upstream and downstream resources, making it easier for customers to get an overview of the information.

(2) The lineage analysis view displays detailed information about other resources related to this resource in a list format:

• Resource name: Click to go to the resource details editing page
• Resource type
• Upstream and downstream: This refers to the direction of data flow and dependency relationships. They are divided into upstream, baseline, and downstream. The currently selected resource is the baseline. Upstream refers to the origin of the data or the starting point where data is generated, while downstream refers to the stages that rely on upstream data for further processing, transformation, storage, or use.

Upstream: data source, visual modeling, dataset
Downstream: dashboard, chart, large screen, visual modeling, dataset

• Hierarchy: Displays hierarchical relationships. The currently selected resource is at level 0
• Resource ID
• Path
• Owner

(2) Click [Access Statistics] to view the access details for this resource:

(3) If the granularity is set to field or dimension/metric, the field information of related resources will be displayed:

(4) To check whether a resource is associated with the selected resource, or when there are many lineage-related resources and you want to view a single resource, enter the Resource ID in the search bar at the top right to search. As shown in the figure below, the data source with Resource ID 354 has a lineage association with the currently selected dataset.

2.4 Download lineage analysis #

Click the download button at the top right to download the current lineage analysis results in xlsx/csv format locally, making it convenient for archiving, further analysis, and presentation.

*Explanation of lineage link completeness and accuracy:

• Dataset: For custom SQL type, only table-level lineage is supported, with 90% accuracy.
• Dimension/Metric: Only the completeness and accuracy of expressions that are data source fields can be ensured.

For example: If there are metrics A1/A2 in the dataset and a field A3=A1+A2 is added, the lineage of A3 cannot currently be identified.