Visual modeling is a data processing and modeling feature provided by DataWind. This article introduces the core workflow for using visual modeling to help you quickly understand the overall process. For detailed instructions on each step, you can refer to several specific documents about visual modeling.

• On the visual modeling page, click "New Task"
• Define the task name, description, and save path. The save path refers to the folder name in the task navigation on the left. If there is no suitable folder, click to create a new folder.
• Task type refers to whether the data source and processing workflow are offline or real-time. Common offline task data sources include Hive; real-time data sources include Kafka.

Note: The task type determines both the types of data connections that can be used and the operator functions available on the canvas

• Click the "+" sign to add a data connection.
• Select connection: You can select an existing dataset, or choose a pre-configured connection from the existing data connections to read data from the corresponding connection (data source)
• If there is no available data connection, click "New Configuration" or use "New Data Connection" to quickly create a data connection.

Before version V2.50, the page appears as follows:

In version V2.50.0 and later, the interface appears as follows:

• Drag the specific table name from the data connection to the canvas
• Select the specific table name and drag it to the canvas to use it

• Click the input node operator, and a drawer will expand at the bottom to provide the data reading configuration page

  The height of the page can be adjusted by dragging the upper boundary of the drawer, or you can scroll to display more content

• Extraction settings: These are the filter parameters for reading the input node's data table. If partitions exist, the partition fields are displayed by default
• Preview settings: This refers to the preview data involved in the canvas execution process (which does not affect the final task execution). In theory, the less preview data, the faster the canvas preview executes. However, some AI operators require more data. It is recommended that the default preview data should not be less than 2,000 rows.

• After completing the configuration, click Execute: This will read the data into the canvas according to the configuration items.
• Preview exploration: This refers to exploring the distribution of each field based on the preview data to determine whether the data is usable.
• Logs: This refers to querying the execution logs of the task. If there is an execution failure, the reason for the failure can be viewed here.

• Perform data cleaning and establish a directed cleaning process.
• Click the plus sign (+) on the output of the input data operator block to expand the available operators. Operator types include: data cleaning, output, feature engineering, machine learning, and natural language processing.
  • Data cleaning: Refer to Data Cleaning. It is mainly responsible for model building (such as multi-table joins and multi-table merges), field format conversion (such as field settings, row-to-column and column-to-row transformations), data computation (such as calculated fields, aggregation, top K values), data filtering (such as deduplication and sampling), and many others.
  • Output: Indicates that after the canvas process is executed, the data is output to the specified location and the task execution logic is configured.
  • Feature engineering/machine learning: Refers to AI algorithm capabilities such as principal component analysis, feature importance, clustering, classification, regression, and many others.
  • Natural language processing: Refers to NLP capabilities, where word segmentation and stop word removal use open-source word lists.

  Feature engineering, machine learning, and natural language processing operators can only be used in conjunction with the "Prediction" operator.

• When adding connections between operators, a connection can only be created if the output node of one operator and the input node of the downstream operator are the same color. In addition, the number of input nodes indicates the number of upstream operators that can be received.
  
• Add operators between connections or delete connections: Hovering over a connection allows you to insert a new operator or delete the connection to establish a new relationship.
  

• Save data to a dataset
• Select the "Output" operator: Decide where to output the data and how the data is output (such as dependency relationships, monitoring alerts, priority, and so on).
• Output dataset: You can select an existing dataset to write the newly created workflow into an existing dataset, or create a new dataset to store the output data.

If you select an existing dataset, you need to confirm whether the data structure matches the required structure. If it does not match, writing may fail.

• Dependency configuration: Establish prerequisite dependency conditions for executing the output operator. For example, if the upstream table is A, after adding the dependency on A, you need to wait until the A task is completed before executing the current operator task.
• Advanced configuration: Configure the priority order for operator task execution and storage partitions.
• Monitoring configuration: Configure rules and notification methods for monitoring alerts.

  To push alert information through the office platform, please first confirm whether "(集成相关,不发布)Office Platform Integration" has been configured.
  

• Workflow execution modes on the canvas
• Manual execution: Task execution requires manual initiation.
• Periodic execution: The system automatically executes tasks according to the specified frequency (such as daily execution, weekly execution, and so on).

After a visual modeling task outputs to a dataset, you can filter the dataset's sources for "visual modeling" to view all tasks output by visual modeling.

• Tasks output to Hive will indicate that they cannot be used for visual queries. Clicking Edit will display the visual modeling task address, and clicking it will take you directly to the details page of the visual modeling task.

• After you click a task that has been exported to ClickHouse, it can be used directly for subsequent visual queries. Clicking Edit will prompt the generation of the visual modeling task address, and clicking this address will take you directly to the details page of the visual modeling task.