Flink is integrated with DataLeap, the big data development and governance suite from Volcano Engine. You can develop tasks in the Flink console and in DataLeap. This article introduces the basic process of developing tasks in DataLeap.

Background #

The big data development and governance suite (DataLeap) is a one-stop big data middle platform solution developed in-house by Volcano Engine. It is designed to solve challenges such as data sharing, data governance, and asset management across various business scenarios and systems. DataLeap helps enterprises improve data development efficiency, reduce O&M costs, discover data value, and support data-driven decision-making.
DataLeap supports multiple computing and storage engine services, including Flink, Batch Computing for Spark, Lakehouse Analytics Service (LAS), and the open-source big data engine E-MapReduce.
To learn more about DataLeap, see Product Definition.

Development Workflow in DataLeap #

1. Create Resources in Flink
   Whether you are developing tasks in Flink or DataLeap, projects and resource pools are essential resources. You need to create projects and resource pools and ensure they are available. To create resources, see Step Two: Import Project and Create VCI resource pool.
   For fine-grained permission control, assign custom permissions to target members within a project to ensure minimal permissions and precise control. For project permission management, see 权限概述.
2. Associate resources in DataLeap
   Create a project in DataLeap and select the Flink project and resource pool to associate. To create a DataLeap project, see Create a project.
   When associating Flink resources, you can only associate one project, but you can associate all resource pools within that project. After associating the resources, you can create and develop tasks.
3. Develop tasks in DataLeap
   You can develop three types of tasks: SQL, JAR, and Pyjstorm. For more information, see:
   • Serverless Flink SQL
   • Serverless Java Flink
   • Serverless Pyjstorm Flink