Data input marks the beginning of users' task processing for visual modeling. Users need to select a data connection to obtain data input from the data source, after which they can choose data cleansing operators or other processing methods.

2.1 Data input processing #

Taking data input from a MySQL data connection as an example, drag the input dataset onto the canvas and click the input operator. The data source information will be displayed, and the extraction method can be set. Non-partition field filtering is supported, with both "AND" and "OR" filter logic currently available. Users can further filter data freely based on the data source. In addition, extraction parameters and the amount of data to preview can also be set.

After completing the basic data input processing above, users can further select operators such as data cleansing or feature engineering to perform diverse data processing and transformation.

2.2 Custom SQL #

Visual modeling supports custom SQL for data input from most offline storage sources, enabling basic data processing. During the data connection stage of task creation, if users select a dataset, the following will be displayed on the left canvas: Custom SQL (available for offline tasks), visual modeling dataset, DataWind dataset, and customer data platform dataset (if the product is purchased and deployed). The introduction to custom SQL is as follows:

• Hive/ClickHouse custom SQL:
  • Hive: When visual modeling outputs are stored as Hive datasets, SQL can be written as long as it complies with Hive syntax.
  • ClickHouse: When visual modeling outputs are stored as ClickHouse datasets, SQL can be written as long as it complies with ClickHouse syntax.

In the custom SQL configuration interface of visual modeling, users can perform operations such as renaming, adding descriptions, copying, and deleting. When used as the first operator node, "Execute this node" and "Execute to this point" are not supported.

A library of common functions is provided in the editing area, including explanations of function usage, command formats, examples, and more.

After editing is complete, the user can set the amount of data for preview calculation, customize the number of rows or use all data, and then click "Data Preview" on the far right to view.