Task creation refers to the process of creating a visual modeling task, which typically includes steps such as creating a new task and establishing data connections. This article introduces the steps for creating an offline task based on the product's actual interface.

Visual modeling tasks support extracting data from data sources, adding data processing nodes via drag-and-drop, and outputting the processed data to the target source.

• Task type: Offline task
• Supported input data sources for offline tasks: Hive, MySQL, ClickHouse, Kafka, HttpAPI, Lark, CSV/Excel, Oracle, Impala, PostgreSQL, Hbase, SQLServer, MaxCompute, ADB, MongoDB, Hana, Teradata, Db2, Vertica, GreenPlum, and more than 20 mainstream data sources
• Supported data cleansing nodes: field settings, row filtering, adding calculated columns, aggregation, join, merge, row-column transposition, and more
• Supported AI modeling capabilities: a variety of operators including featured zone operators, feature engineering, machine learning, and natural language processing
• Supported output built-in data sources: datasets stored in Hive, ClickHouse, and ByteHouse

The trading calendar and multiple synchronization features in Save Task - Run Type - Run Frequency are currently only supported in privatized deployment products version V2.83.0 and above. To enable this feature, please contact the BytePlus product team.

After entering "Visual Modeling", click "Task List" — "New" — "Create Offline Task" to navigate to the task creation page.

On the new task page, click the plus sign in the upper left to add a data connection. Multiple types of data connections can be added, and the platform supports custom SQL for most offline storage types.

If a dataset is selected, the following will be loaded on the left canvas: custom SQL (displayed for offline tasks), visual modeling datasets, intelligent data insight datasets, and customer data platform datasets (if the product is also purchased and deployed).

• Hive/ClickHouse custom SQL:
  • Hive: Visual modeling outputs and stores data as Hive-type datasets. SQL can be written as long as it complies with Hive syntax.
  • ClickHouse: Visual modeling output with data stored as a ClickHouse dataset. You can write SQL as long as it complies with ClickHouse syntax.
• Visual modeling dataset: The dataset output from visual modeling, regardless of whether the output storage type is Hive or ClickHouse.
• Intelligent Data Insight dataset: The dataset output built through the "Dataset" module.
• Customer Data Platform dataset: If the Customer Data Platform is also purchased and deployed, this type of dataset can be used here.

In the data connection list on the visual modeling task creation page, click the delete button to the right of a specific data connection to remove it.

Click the plus sign on the right side of a node to add and configure a processing node. Drag a line from the plus sign on the right of the previous node to the origin on the left of the next node to configure the node flow relationship. After clicking "Apply", you can expand to preview the processed data results.
As shown in the figure below, click the output plus sign of the input data operator block to expand six types of operation nodes: Output, Data Cleaning, Featured Zone Operators, AI-Feature Engineering, AI-Machine Learning, and AI-Natural Language. Click one of these types to see the available operators.
Operator refers to a data processing node.

Description of common operators

• Data Output: Indicates that after the canvas workflow is executed, data is output to the specified location and the task execution logic is configured.
• Data Cleaning: Mainly responsible for model building (such as multi-table joins and merges), field format conversion (such as field settings, pivoting rows to columns and vice versa), data computation (such as calculated fields, aggregation, top K values), and data filtering (such as deduplication and sampling), and more.
• Featured zone operators: Includes large model operators such as Volcano Ark large models.
• Feature Engineering/Machine Learning: Represents AI algorithm capabilities such as principal component analysis, feature importance, clustering, classification, regression, and more.
• Natural Language Processing: Represents NLP natural language processing capabilities. For tokenization and stop word removal, open-source word packages are used.

• Feature engineering, machine learning, and natural language processing operators can only be used in conjunction with the "Prediction" operator.

Operators are not only visible on the canvas, but can also be accessed by clicking "Operator" in the tab bar on the left to select available operators. The application method for each operator and model differs; detailed introductions are provided in the corresponding operator documentation. This article only introduces the main process for creating a task.
In actual use, visual modeling operators support adding text descriptions to help users better understand the remarks for the current operator.

The visual modeling task creation page supports switching between free layout and grid layout, allowing flexible adjustment of the layout for each task node.

The visual modeling new task page supports unified display of abnormal nodes in the task, assisting in troubleshooting task issues.

The visual modeling task creation page also supports deleting operators by pressing the delete key, improving data cleaning efficiency.

View preview #

1. On the visual modeling task editing page, after selecting a data connection, convenient preview is available. You can preview detailed data, table structure, and data profiling.
   
   • You can set whether to expand the data preview panel for the current type of operator by default and apply this setting to all operators to suit different user preferences.
   • The preview panels for "data profiling" and "detailed data" are merged to provide centralized information display, simplify operations, and improve feature usage.
2. Click the "▼" button for each column of data to quickly select an operator for data processing. As shown in the figure below:
   

Apply suggestions #

In the preview, data cleaning suggestions are provided. Users can clean the data with one click, improving the efficiency of data cleaning.

Instant settings #

On the visual modeling task editing page, when opening data preview, you can instantly edit or configure the preview in the lower left corner of the preview interface. Users can configure the amount of data involved in the preview calculation (by default, no more than 1,000,000 rows), and can customize the number of rows or select all data. After configuration, the preview will be refreshed in real time. As shown in the following figure:

In the configuration, you can select a node and run the node to perform an execution check. Caution: When an upstream node is modified, you must rerun the upstream node before you can execute the current and downstream nodes.

• Run this node: Runs the current node; all upstream nodes must have been executed.
• Run to this point: Sequentially runs any upstream nodes that have not been executed and the current node.
• Run from this point: Sequentially runs the current node and subsequent nodes; all upstream nodes must have been executed. This is typically used after the current node is modified.

After a node is executed, a green check mark will appear next to the node, indicating that the current node has been executed.

Add an output node and choose to output to an existing dataset or create a new dataset. For details on output dataset settings, see Data Output.

Now supported: datasets stored in Hive, ClickHouse, or ByteHouse Data Output

1. After building the visual modeling task, you can click "Test run" to verify the execution result.

2. After the test is successful, you can open the "Edit task information" dialog to set the task name, description, path, and run type.
   
   The parameter configuration instructions are as follows:

   Parameters	Configuration instructions.
   Name	Visual modeling task name.
   Description	Description of the visual modeling task, which can include information such as the task objective, task content, and other information.
   Path	You can choose to save it to a specific task list folder for easier access later.
   Run type	You can set the task execution cycle. Manual run: The system will not run automatically. The user needs to manually select the task in the task list and click Run. Scheduled run: The system automatically runs the visual modeling task according to the set schedule, without requiring manual execution by the user.

   Periodic run—The run frequency can be configured in two ways, as described below:

   Exact time.

   Interval duration.

   Includes hourly, daily, daily (based on trading calendar), daily (the day after trading calendar), weekly, and monthly levels. Supports setting multiple times. When multiple times are selected, the schedule will execute multiple times. For example, if 9:00 and 10:00 are set for each day, the task will execute twice daily. For daily (based on trading calendar) and daily (the day after trading calendar), selecting the trading calendar type is supported. For trading calendar settings, see BytePlus "System Configuration" - "Enterprise Calendar" - "Tag Management".

   Supports execution every 10, 20, or 30 minutes. Supports setting the effective time for each day, which must include both a start time and an end time. If left blank, the effective time is the entire day. warning Full synchronization or large-volume synchronization using minute-level synchronization will consume significant resources and result in poor performance. Use as needed.

   warning

   • After modifying the run frequency, check whether the output-dependency configuration (such as dependency information) needs to be updated accordingly.
   • The trading calendar and multiple synchronization features in Save Task - Run Type - Run Frequency are currently only supported in on-premises deployment products version V2.83.0 and above. To enable this feature, please contact the BytePlus product team.

3. When the model configuration is complete, you can click Save or Save as in the upper right corner. If the model is not complete, you can also set the run frequency to manual and then click Save or Save as.
   

4. After saving the task configuration, click Return in the upper left corner to go to the task details page.

5. Return to the visual modeling page and click "Run records" for the current task to run it manually. For more task management capabilities, see Task Management.
   

On the dataset details and data preview pages, click "Download" to download the data. The supported data formats are CSV and Excel. The number of data entries includes several preset options: 100, 500, 1000, 5000, and the current previewed data. Customizing the number of data entries is also supported.