1.1 Common error messages #

What could cause synchronization to fail after modifying the field type of a hive table? #

Example 1: The hive data is not empty in the original database, but after synchronizing to DataWind, both dataset preview and visual query show only null values.
The database contains values:

After dataset synchronization, the preview values are null:

Example 2: The hive data was initially synchronized to DataWind as int type. It was found that the data type did not meet expectations (all values displayed as 0, expected value was 0.23). After changing the data type in hive to double, the DataWind data preview still did not show the expected 0.23.
Root cause analysis: A hive table consists of a storage file and a metadata file. When modifying the field type, only the metadata file was changed, while the storage file type remained unchanged, resulting in abnormal type conversion during data synchronization.
Solution:

• After modifying the field type of the hive table, reload the data into the hive table.
• Then, in DataWind, edit and save the corresponding dataset, and resynchronize the data.

Kafka dataset integration reports OLAP interface exception #

Possible cause: Field format is not allowed
Troubleshooting suggestion:
Currently, nested json is not supported, and modifying map field columns is not supported.
If the issue is not as described above, please refer to the following documentation for troubleshooting Kafka integration issues: Kafka.

Lark spreadsheet data source shows "upstream data source token does not exist". #

Possible cause: This issue may occur the first time a user uses the Lark data source, due to a missing Lark spreadsheet permission token.
Solution:
The frontend page will display a QR code. The user can scan the code to obtain the token.

1.2 Common issues with data definitions #

Data accuracy and definition issues #

Similar issues:

The number for XXX is incorrect
Data for the same dimension is inconsistent across different tables
Links in charts or dashboard tables cannot be opened
How to submit a data request
Incorrect metric

Solution:
To understand the department's data situation or to request data, it is recommended to contact the department's analyst or data developer for confirmation;
For issues related to data table accuracy or data definition, it is recommended to contact the corresponding dataset owner for confirmation.
For issues with content display in charts or link redirection, it is recommended to contact the corresponding dataset owner or relevant business for confirmation.

1.3 Common issues with data integration #

Format error prompted when uploading dataset #

Please check whether the uploaded dataset contains line breaks. Line breaks will be recognized as multiple data entries, which may result in format errors.

Lark spreadsheet upload failed / Data does not match expectations #

1. Permission-related (Common errors: Invalid connection information: Invalid link; No permission)
(1) The person scanning the code must have read permission for the provided link.
(2) Providing a single link will import all sheets under that document.
2. File requirements
(3) Only standard spreadsheets are supported for recognition.
(4) The document must be named and cannot be "Untitled Spreadsheet". The document and sheet names must not contain the English period '.', otherwise the document schema may fail to be retrieved.
3. Document format requirements
(5) Only the first group of columns with consecutive non-empty column names will be read; the first column name cannot be empty, otherwise an error will occur.
(6) Column names should preferably be plain text, must not be all numbers, images, or hyperlinks, must not be named p_date, and must not contain $, `(backtick), or !, otherwise synchronization may fail. After creating the dataset, do not arbitrarily modify the column names in the Lark spreadsheet, otherwise synchronization may fail. Column names that are too long may cause synchronization to fail.
(7) Column names must not be duplicated, otherwise an error will occur.
(8) Rows should preferably not be empty. Completely empty rows are counted in the sheet's row count and will be added to the data in a format like [null, null, null, null].
(9) Merged cells and formulas are not supported.
(10) Try to avoid having different data types in the same column, otherwise synchronization may fail.
Common error causes:
Invalid connection information: Invalid link; No permission; Untitled Spreadsheet; Token expired
Schema error: First column name is empty; Invalid column name format

Uploading a Lark spreadsheet displays 'application illegal.' #

Lark spreadsheets are supported for creating datasets.
To enable Lark datasets, you must first complete Basic Configuration for Lark Integration configuration.

1.4 Common issues in dataset creation #

What is p_date #

p_date is the partition date field of the dataset. The product will rename the date/p_date partition fields in the original Hive table to p_date.
p_date represents the data generation date in the upstream Hive table, or the date when data is synchronized to the dataset (if the upstream data does not have a date partition). In visual queries, p_date can be used to select and filter by the date dimension according to analysis requirements.

How to analyze offline data #

Select dataset - add dataset, and choose the corresponding data source. To analyze offline data, upload offline data (Excel and CSV are supported) as the data source. See Local Excel/CSV Files.

Unable to create the corresponding dataset after applying for table permissions with row restrictions #

After applying for the corresponding rows, add row restrictions in the filter conditions when creating the dataset. If only the data permission for app_name='news' was applied for during permission application, restrict the filter condition to app_name='news' when creating the dataset.

How to add Lark spreadsheets #

Similar issues:

Online reporting

Operation steps:
Create a spreadsheet in Lark, go to the dataset interface, click the left sidebar'Create dataset' — select 'Lark spreadsheet' — enter the link of the document to be uploaded. See Lark Sheets.

Why is the data table not found in search #

The product only displays tables that the currently logged-in user has permission to view. If the table cannot be found, check whether the current data connection or login account has read permission for the corresponding table.

1.5 Common issues in data synchronization #

The dataset keeps showing 'waiting for execution' #

Similar issues:

Synchronization is not triggered after upstream data processing is completed
Dataset synchronization is stuck
Dataset synchronization is blocked
The scheduled dataset synchronization task has not started yet

There are numerous projects within the product, and each project contains datasets with corresponding synchronization tasks. All synchronization tasks are randomly queued together to use queue resources.
Data synchronization process:
At the scheduled time—all scheduled tasks within the product begin queuing to submit yarn tasks—yarn queue starts queuing—calculation and scheduling are officially executed—synchronization succeeds
If concurrency exceeds the limit during the same period, data will remain in a waiting-for-execution state.
Ways to accelerate:

• Adjust dataset priority. Priority affects the order of scheduling; the higher the priority, the easier it is to obtain resources.
• Set a scheduled time during non-peak hours. For example, 8:00–10:00 daily is peak time, so you can avoid this period.

Dataset synchronization is slow #

Possible causes:

• Incorrect dataset configuration
• Unreasonable parameter configuration

Troubleshooting solutions:

• Verify whether the model configuration is correct, including connection method, join conditions, and more, see Model Configuration.
• Optimize synchronization parameter configuration to improve synchronization efficiency. See Extraction Parameters.

Lark table synchronization failure #

Similar issues:

Why does synchronization fail after modifying the Lark table?
Why is the modified Lark table dataset unable to synchronize?

Possible causes:
Deleting or renaming existing fields prevents synchronization from retrieving information about those fields.
Recommended solutions:
On the model configuration page, check whether the currently selected fields match the column names in the Lark table.
If there is a mismatch, click Edit to enter the dataset model configuration page and select the new fields.
Then match them on the field configuration page.

The dataset shows waiting for execution and does not synchronize. #

When the number of synchronization tasks within the product exceeds the concurrency limit, some tasks will be queued and display a waiting-for-execution status during synchronization. Please wait patiently. To increase task priority, adjust the scheduled synchronization time or the priority.

What should I do if data exceeds the limit? How to modify the lifecycle #

Path: Data management - Details - Synchronization settings - Settings - Edit
Lifecycle limitation: For a single partition, if the data volume is less than 30T, the lifecycle can be set to 1-1500 days!

Historical data in the dataset disappears #

Please check the lifecycle configured for the dataset. After expiration, partition data will be deleted; however, synchronization records will still be retained.

How to add a URL link description for a dataset #

Select the dataset, click ⋮, and choose Create Example to add a link description.

When is the previous day's data updated on the next day #

The dataset will be synchronized according to the configured synchronization time. If synchronization is set for 00:00 every day, updates will start at 00:00.

When saving the synchronization configuration, if the data volume exceeds the limit and you are prompted to reduce the data range or shorten the retention period, what should you do #

Change the lifecycle to 7 days. After creation, go to Synchronization status to modify the required lifecycle.

1.6 Common issues with editing, modifying, or deleting datasets #

After updating fields in the base table, the dataset is not updated #

Similar issues:

Fields have been added to upstream data, but they do not appear in the dataset
After modifying the field type in the base table, the field type in the dataset is not synchronized
After the underlying table is updated, there is no data in visual queries
Newly added fields in the dataset are not visible

After adding or modifying fields in the underlying table, update the selected fields in the dataset model configuration. After updating, resynchronize the dataset so that the new fields contain data. Once the new fields for the relevant date contain data, perform the corresponding visual queries.

Can the dataset associated with an existing dashboard be modified #

1. Non-performance settings of the dataset can be modified directly. After modification, wait for execution to complete before querying in the dashboard (historical data must be resynchronized manually).
2. Performance settings of the dataset cannot be modified
3. When changing fields, ensure that the field names used in the dashboard remain unchanged to avoid query failures.

If a dataset is deleted, will the charts remain #

After deleting a dataset, the chart is simultaneously removed from the chart list page. The chart still occupies its position in the dashboard, but a prompt appears stating "No record found in the database".
After restoring a dataset from the recycle bin, the chart is restored in the chart list. The chart continues to occupy its position normally in the dashboard.
If a chart is deleted from the chart list, it is permanently deleted. The chart disappears from the dashboard, which affects the layout.

It is not possible to save after deleting a field from the dataset. #

Please check whether: 1. The deleted field is a join field; replace the join field. 2. Other fields depend on the deleted field; delete fields that depend on the deleted field, or modify the dependencies.

1.7 Other common issues in dataset management #

How to categorize fields after creating a category #

Go to the corresponding dataset, click to modify the field, and select a category for the fields that need to be categorized.

How to view the dataset owner in the visual query interface #

Click the ⋮ next to the dataset name and select View Dataset to see the dataset details and owner.

How to check the base table name of a dataset #

Go to the visual query for the dataset, click the query history in the upper right corner, and select the SQL details of any historical query to view the ClickHouse table name corresponding to the dataset.