In the dataset list, the operation items for an individual dataset include the visual query configuration for the dataset. Intelligent Data Insight allows you to visually configure certain dataset properties for frontend query analysis. This feature is mainly related to query performance when there are many datasets. It is generally recommended to enable it when queries are slow, and dataset administrators or data personnel should make this determination.

(1) Enter your project;
(2) Click the Datasets tab;
(3) Click the configuration item for the corresponding dataset in the dataset list (as shown in the figure);
(4) Click "Visual Query Configuration";

(5) As shown in the figure below, you can configure push control, query control, and more settings as needed.

3.1 Push control #

3.1.1 Dataset subscription #

• Datasets can generally be subscribed to in analysis queries. When the data itself is a large volume of detailed records, it is recommended to disable this feature (that is, select Not Allowed).
• Once Not Allowed is selected, the dataset will not participate in subscription queries, and the chart data based on this dataset will not be displayed in dashboard subscription screenshots. However, it can still be displayed normally when the dashboard is actually opened. As shown in the figure below, the dataset for the chart on the left has subscription set to [Not Allowed], while the dataset for the chart on the right is set to the default [Allowed].
• Selecting [Not Allowed] for subscription does not affect the use of the dataset in visual queries or the normal viewing of specific data in the dashboard.

3.1.2 Create monitoring #

• Datasets are generally allowed to be monitored in analysis queries, but when the data itself is a large volume of detailed records, it is recommended to disable this feature (that is, select Not Allowed).
• Once 'Not allowed' is selected, the dataset will not participate in creating new monitoring, and the [Create Monitoring] feature for charts built on this dataset will be unavailable. As shown in the figure below, the chart dataset on the left has 'Not allowed' selected for new monitoring, while the chart dataset on the right is set to the default 'Allowed'.
• Selecting 'Not allowed' for creating new monitoring does not affect the use of the dataset in visual queries or the normal viewing of specific data in dashboards.

3.2 Query control #

3.2.1 Automatic query #

• The [Automatic Query] feature for visual queries corresponding to the dataset is enabled by default. If the dataset contains a large volume of detailed list data, it is recommended to disable this feature (that is, select 'Not allowed').
• After selecting 'Not allowed', automatic query will be disabled by default during visual queries. Data will only be queried from the dataset after manually clicking the query button.
• Disabling automatic query does not affect dashboard presentation.

3.2.2 Filter search optimization #

• The scope of this feature includes Dimension Filter and Filtering.
• When set to 'Allowed', the filter will prioritize using frontend cache when searching for data items in the exact search list, resulting in higher filtering efficiency.
• When set to 'Not allowed', searching for data items in the list will query all backend data from the past 30 days to ensure data integrity. If the partition does not cover 30 days, the actual number of days will be used.

3.2.3 Query date range limit #

Users can set a query date range limit for the dataset. After setting this limit, dataset users can only query data within the specified date range.
Note: If the dataset in use has already been configured with a 'maximum query date range' limit for visual queries, the filter will also take effect synchronously when querying and configuring filters based on this dataset in visual queries. When the maximum query date range set for the dataset is less than or equal to the number of days for displaying only the most recent N days of sampled data, only the most recent N days of sampled data will be displayed.

3.2.4 Detail fields #

Supports setting fields in the dataset as detail fields. When no metrics are added to the query, these fields can only be used for detail queries.

Click to switch to the detail table for querying.

3.3 More #

• Default visualization query

Supports setting a default visualization query link for the current dataset. After configuring the default visualization query link, when users open the visualization query for the dataset, they will automatically enter the configured default query.

3.4 Timeout nodes #

• When querying a very large amount of data, the data return time can be long. If the accuracy requirement for the data is not high at this time (for example, if the summary statistic result is 100, it is acceptable to return a result of 99.98), you can set to allow discarding timeout nodes.

• After choosing to discard timeout nodes, you need to set the timeout duration and error ratio.
  • Timeout duration: The return time for data queries at each data query node. The default value is 100 ms and can be adjusted as needed. If the query time for a node exceeds this duration, the data result from that node will not be returned.
  • Error ratio: For example, 0.100. If the number of nodes exceeding the above duration divided by the total number of query nodes is greater than or equal to 10%, the dataset query will report an error and no data will be returned. If the number of nodes exceeding the above duration divided by the total number of query nodes is less than 10%, the dataset query will return data normally. Caution: The precision of this ratio is to three decimal places.
• By default, discarding is not enabled, and this will not affect the data results in visualization queries and dashboards.

3.5 Impact on historical stock resources #

3.6 Long-period query configuration #

• When querying data over a long time period, you can use the Presto engine to query Hive tables.

3.7 Null join #

• If there are doubts about the visualization query results, when the current query includes LOD, visualization join, period-over-period, or comparison functions, a prompt about null values will be displayed. Click "Go to enable" to close the current troubleshooting pop-up and open "Query configuration".
• In the "Query configuration" window, you can configure cache, sampling, and join settings.
• Null join explanation:
  • When the current query uses visual join, period-over-period comparison, comparison, or when a field involved in plotting is a lod field, Null join configuration is supported.
    • Null join configuration only supports extract-type datasets.
    • The configuration for "join" is displayed and is not selected by default.
    • When the above conditions are no longer met, this option is grayed out and cannot be selected.

• Why restrict whether users are allowed to enable these basic features of "subscription" and "create monitoring" on datasets?

--Because some users' datasets are extremely large and queries perform poorly; at the same time, since subscription and monitoring periodically scan all data, they have a significant impact on the data storage and computing engine. For example, some complete advertising detail lists are used in subscription or monitoring, but such scenarios are not recommended for these features. Therefore, for such large-scale detail list data, feature switches are provided to control whether subscription and monitoring are allowed, so that users can adjust according to actual needs.