Docs

Database Transmission Service

PostgreSQL connected over public network as the source

Migrating to PostgreSQL connected over express network

This topic describes how to create a task for migrating the PostgreSQL connected over public network to the PostgreSQL connected over express network on the Database Transmission Service (DTS) console.

Prerequisites

A self-managed instance and its database for PostgreSQL have been set up in the public network environment.

Set the target information as needed, as detailed below:

When using VPN to perform data migration, the local network where the self-managed database is located has been connected to Volcano Engine via VPN gateway. For detailed steps, refer to Building a cloud VPC and on-premises multi-data-center network interconnection.

If data migration requires a dedicated line connection, you need to establish a dedicated line connection between a single cloud VPC and a single on-premises data center network. For more information, refer to Configuring direct connections.

If you need to migrate data from the PostgreSQL engine instance managed in the AI native database management platform (AIDAP):

AIDAP-managed Supabase engine instance:

Create workspace. For detailed steps, refer to Create workspace.

Create database branch. For detailed steps, refer to Creating branch.

Obtain the connection information required by DTS, such as host address, port, account number, and password. refer to Obtaining the connection information of computing power for detailed steps.

AIDAP-managed standard PostgreSQL engine instance:

Create workspace. For detailed steps, refer to Create workspace.

Create database branch. For detailed steps, refer to Creating branch.

Obtain the connection information required by DTS, such as host address, port, account number, and password. refer to Obtaining the connection information of computing power for detailed steps.

The network connectivity and service availability of the source and destination have been confirmed.

Optional. Check each migration object in the source and target databases according to the instructions in the pre-check items. For more information, refer to Pre-check items (PostgreSQL).

Precautions

When the source is a PostgreSQL connected over public network, pay attention to the following information:

If the source switches to the primary during migration, the migration task will fail.

If the source end performs a restart operation during migration, the migration task will be temporarily interrupted and automatically attempted to recover. If the task does not recover for a long time, submit a ticket for technical support.

To ensure real-time data consistency, it is recommended to select schema migration, full data migration, and incremental data migration.

Only data from the primary database supports migration. Data from the secondary database is not supported.

The migration object granularity can be either database or table. If tables are selected as the migration objects, other objects such as views, triggers, functions, or stored procedures will not be migrated to the target database.

For PostgreSQL migration tasks, the source database account must create triggers, functions, and other objects in the source database. Therefore, during the migration process, do not modify the access permissions of objects such as triggers, functions, or internal tables created by this account in the source database.

Since full data migration performs concurrent insert operations, the collection in the target database generates fragments. Therefore, the storage space of the collection in the target database will be larger than that of the collection in the source database after the full migration is complete.

The current data migration through dedicated lines is in beta testing. You can submit a ticket to contact technical staff to implement data migration.

During migration, if there are CREATE TABLE or RENAME TABLE operations on the source end, select the entire database containing the table or the entire database where the table is renamed to when configuring the migration object. Otherwise, the newly created or renamed table will not migrate to the destination.

To ensure data migration performance and task stability, ordinary triggers, event triggers, RULE, exclusion constraints, foreign key constraints, check constraints, constraints on DOMAIN, and Sequence value settings in the source database will migrate only after incremental migration completes.

During the pre-check, if you ignore the alert and start the migration task, it may lead to data inconsistencies and pose business risks.

When creating a data migration task via express network, the system creates several network cards by default in your selected VPC, and the network cards are mounted by default to the subnet you selected. The network card name format is dts_shuttle_********.

During data migration, if duplicate primary key values or unique key values are encountered in the target database, DTS will not retain the conflicting data at the destination. Instead, the data from the source will overwrite the data at the destination. When a conflict overwrite occurs, if there is a table with the same name in the target database, it may cause data inconsistency issues as detailed below. It is recommended to use table name mapping or delete or rename the table with the same name in the target database to avoid data inconsistency and reduce business risks.

When the table structure is consistent, the migrated data overwrites records in the target table with the same primary key or unique key values as in the source database.

When the table structure is inconsistent, initialization of data may fail, and only partial column data can be migrated.

Limits

Application limits

Type	Description
Source	Bandwidth requirements: Adequate egress bandwidth is required on the source database server. Failure to meet this prerequisite will lead to suboptimal migration performance. Tables to be migrated must have a primary key or a unique non-null constraint, and their fields must be unique. Otherwise, data inconsistency may occur. The database or table name of the object to be migrated can not contain question marks (?). Otherwise, the migration task will fail. It is recommended that the number of tables in a single task should not exceed 20,000, and the number of schemas should not exceed 50. When a large number of databases and tables need to be migrated, it is recommended to split them into multiple tasks. WAL log parameters required for incremental migration: You must enable WAL logs, that is, set the value of the wal_level parameter to logical. WAL logs must be retained for at least 24 hours, and a retention period of more than seven days is recommended. Otherwise, the DTS task may fail because it can not obtain the logs, potentially leading to data loss.
others	A single migration task can only migrate one database. To migrate multiple databases, configure a migration task for each one. It is recommended that the source and target PostgreSQL databases be the same version, or migrate from a lower version to a higher one to ensure compatibility. If you migrate from a higher version to a lower version, there may be database compatibility or data loss issues. During the incremental data migration process, if the granularity of the migration object is schema, ensure that the table-level replication identity REPLICA IDENTITY is set to FULL. If a new table is created in the schema to be migrated, or the table to be migrated is reconstructed using the RENAME command, you need to execute the ALTER TABLE schema.table REPLICA IDENTITY FULL; command before writing data to the table. tip When executing the ALTER TABLE schema.table REPLICA IDENTITY FULL; command, replace schema and table with the actual schema and table names. During incremental data migration, DTS creates a replication slot with the prefix dts_sync_ in the source database to replicate data. DTS automatically cleans up historical replication slots every 5 to 10 minutes to prevent them from continuously accumulating and taking up disk space. tip When a migration task is released or fails, DTS actively cleans up the replication slot. If a primary/secondary switchover occurs in RDS for PostgreSQL, you need to log in to the secondary database to manually clean up the replication slot. Prior to executing data migration, assess the performance of both source and target databases. It is also advised to perform the migration during off-peak business hours. Otherwise, during full data migration, DTS consumes certain read/write resources on both source and target databases, which may lead to an increase in database load. Since full data migration performs concurrent insert operations, the table in the target database generates fragments. Therefore, the storage space of the table in the target database will be larger than that of the table in the source database after the full migration is complete. Verify that the character sets of the source and target databases are consistent before migration. Otherwise, pre-check may fail.

Operation limits

During the schema migration and full migration stages, do not perform DDL operations that change the database or table structure, or the data migration task will fail.

If only full data migration is performed, do not write new data to the source database, or the source and target data will be inconsistent. To maintain real-time data consistency, it is recommended to select schema migration, full data migration, and incremental data migration.

Minimum database permission policy

The account used for data migration must have the following permissions:

The source database account needs to have superuser permissions to create logical replication slots.

The target database account needs to have permissions to create a schema. If the target database schema already exists, the migration account must be the target schema owner.

On the Create migration task page, locate the Configure source and target database wizard, and configure the following parameters.

Type	Name	Description
Basic information	Task name	Optional. Customize the task name. The length must be within 256 characters. tip If not set, the system generates a task named with dts- by default.
Basic information		Link specification	Select the link specification for the migration task. Compact, Medium and Standard are supported. tip The link specification can not be modified after the task is successfully created. For details on the link specifications, refer to Product specifications.
Source database configuration	Select an existing data source	You can choose whether to use an existing instance as needed: Use existing instances: Click Select an existing data source, then in the Select data source control panel, choose the target region from the Region drop-down list, and select the target data source. Supports selecting data sources by data source name and ID. For methods to create data sources, refer to Data source management. Do not use existing instances: If not using an existing instance, you need to configure the following database instance information.
		Instance type	Select PostgreSQL.
		Access method	Select PostgreSQL connected over public network. tip When migrating PostgreSQL connected over public network instances, add the DTS IP segment access permission in the source end's allowlist or security group policy in advance. Click View DTS CIDR blocks to view and copy the CIDR block.
		Region	Select the region where the target instance is located.
		Host address	Enter the host address of the target self-managed instance.
		Port	Enter the port number of the target self-managed instance.
		Database name	Enter the database name.
		Database account	Enter the database account of the target instance.
		Database password	Enter the password corresponding to the database account of the target instance.
		Connection method	Select whether to enable the encryption protocol as needed to establish a secure connection between the client and the database instance, preventing data interception, tampering, or forgery during transmission. Option: Unencrypted connection: suitable for self-built PostgreSQL instances deployed in a public network environment. This method carries security risks and is recommended only for testing environments or controlled networks. SSL secure connection: Recommended for production environments, supports connections to PostgreSQL instances hosted on the AI native database management platform (AIDAP), and can also be used for self-built instances configured with SSL.
		Test connection	Click Test connection to verify the connection to the instance. If the test connection fails, modify the configuration based on the failure prompt, and then click Test connection again to verify the connection. The specific prompts are as follows: Connection authentication failed. Check the network: When this error prompt appears, confirm network connectivity. Connection authentication failed. Check the username and password: When this error prompt appears, verify your username and password. Invalid IP address: When this error prompt appears, verify the accuracy of the IP address.
Target database configuration	Select an existing data source	You can choose whether to use an existing instance as needed: Use existing instances: Click Select an existing data source, then in the Select data source control panel, choose the target region from the Region drop-down list, and select the target data source. Supports selecting data sources by data source name and ID. For methods to create data sources, refer to Data source management. Do not use existing instances: If not using an existing instance, you need to configure the following database instance information.
		Instance type	Select PostgreSQL.
		Access method	Select PostgreSQL connected over express network.
		Region	Select the region of the virtual private cloud.
		Host address	Enter the host address of the target self-managed instance. tip Currently, only IPv4 addresses are supported for access. IPv6 addresses are not yet supported.
		Port	Enter the port number of the target self-managed instance.
		Database name	Enter the database name of the target self-managed instance.
		VPC	Select the target VPC from the drop-down list as follows: Direct connect: Select any VPC under the CEN when achieving data connectivity through a dedicated line CEN. VPN connection: Select the VPC bound to the VPN gateway when achieving data connectivity through VPN.
		Subnet	Select a subnet within the VPC from the drop-down list. tip When migrating a PostgreSQL instance connected over express network, you need to add subnet CIDR block access permissions to the allowlist or security group policy on the source end in advance. Click View subnet CIDR blocks to view and copy CIDR blocks.
		Database account	Enter the database account of the target instance.
		Database password	Enter the password corresponding to the database account of the target instance.
		Connection method	Select whether to enable the encryption protocol as needed to establish a secure connection between the client and the database instance, preventing data interception, tampering, or forgery during transmission. Option: Unencrypted connection: Suitable for self-managed PostgreSQL instances deployed in a virtual private cloud environment. This method carries security risks and is recommended only for testing environments or controlled networks. SSL secure connection: Recommended for production environments, supports connections to PostgreSQL instances hosted on the AI native database management platform (AIDAP), and can also be used for self-built instances configured with SSL.
		Test connection	Click Test connection to verify the connection to the instance. If the test connection fails, modify the configuration based on the failure prompt, and then click Test connection again to verify the connection. The specific prompts are as follows: Connection authentication failed. Check the network: When this error prompt appears, confirm network connectivity. Connection authentication failed. Check the username and password: When this error prompt appears, verify your username and password. Connection authentication failed, unable to create connection normally, contact customer service: When this error prompt appears, submit a ticket to contact technical staff. Invalid IP address: When this error prompt appears, verify the accuracy of the IP address. The Express network requires the subnet to have * available IPs. The current number of available IPs in the subnet is insufficient, and the task can not be created. Check the subnet resources in the VPC: When this error prompt appears, adjust the number of available IPs in the VPC subnet to ensure enough IPs in the target VPC subnet.
None	Project	Optional. Select the affiliated project for the task from the drop-down list. By default, it is under default (default project). You can also click Create new project to create a new project. For more information, refer to Creating a new project. tip To grant sub-user privileges to access resources in a specific project, refer to Modifying project privileges. After a task is created successfully, if you need to change its affiliated project, you can move the resource to another project. For more information, refer to Moving resources to a project. A project is a virtual concept that includes a set of resources, users, and roles. A project allows for a unified view and management of a set of resources, and controls the permissions of users and roles within the project for those resources.
		Tag	(Optional) Click Add tag to add a tag to the instance. Tags can help you categorize cloud resources, control access to them, and allocate bills and costs. A tag is a key-value pair that identifies a cloud resource. You can use tags to categorize resources with the same characteristics from different dimensions, such as purpose or owner, making it easier to filter, aggregate, and manage your cloud resources. For more details about tags, refer to Tag overview. The rules for tag key-value pairs are as follows: Key Supports any language, characters, numbers, spaces, or symbols (_.:/=+-@). Case-sensitive, and the length is 1 to 128 characters. In UTF-8 encoded Unicode format. Keys cannot start with any form of volc:, such as Volc: or VOLC:. (Optional) Value Supports any language, character, digit, space, or symbol (_.:/=+-@). Case-sensitive, and the length is 0 to 256 characters. In UTF-8 encoded Unicode format. tip Support fuzzy search for selecting existing tag keys and tag values. A maximum of 20 tags can be associated with a task in a single operation. A maximum of 50 tags can be associated with a single task. For a single resource, tag keys must be unique, and each tag key can have only one value.
		Billing method	Currently, only Pay-as-you-go is supported. No selection is required. tip For detailed billing information, refer to Billing instructions.

Click Next step: configure migration object.

On the Configure migration object wizard page, configure the related parameter information as shown in the table below.

tip

If you do not need to start the migration task immediately, click Save in the lower-right corner of the page after configuring the migration object.

After saving, you will be redirected to the Migration task list page. The task status will be To be started, and you can configure the migration task at a suitable time. For configuration methods, refer to Configuring a migration task.

Name			Description
Migration type			Select the migration type as needed. Currently supported types are Schema migration, Full migration, and Incremental migration. including: If only full migration is required, select both Schema migration and Full migration. If online migration is needed, select Schema migration, Full migration, and Incremental migration simultaneously. tip When Incremental migration is not selected, do not write new data to the source database instance during data migration to ensure data consistency. To ensure real-time data consistency, it is recommended to select schema migration, full data migration, and incremental data migration.
Migration object			In the Migration object area, check the objects to migrate. You can confirm the objects to migrate in the Selected object area. tip The migration object granularity can be either database or table. If tables are selected as the migration objects, other objects such as views, triggers, functions, or stored procedures will not be migrated to the target database. To ensure data migration performance and task stability, TRIGGER, RULE, FOREIGN KEY, SEQUENCE, and other objects of RDS for PostgreSQL will migrate only after the incremental migration completes. The search box in the Selected object area supports fuzzy search to batch delete unwanted targets.
Advanced configuration	Enable millisecond-level delay monitoring	When Migration type selects Incremental migration, millisecond-level latency monitoring is enabled by default. Enabling this feature achieves high-precision, full-link, proactive end-to-end latency measurement, effectively improving stability and detecting anomalies such as network interruptions and idle disconnects. With this feature enabled, DTS writes heartbeats to the source database, requiring the source account to have SELECT, INSERT, and UPDATE permissions. For more information on heartbeat tables, refer to Manual creation of heartbeat tables.
		Skip post-schema migration	When selecting schema migration in migration type, DTS automatically migrates post-schema after the incremental migration ends. You may also skip post-schema migration, meaning DTS will not migrate post-schema objects. RDS for PostgreSQL database post-schema migration objects include ordinary trigger, event trigger, RULE, exclusive constraint, foreign key constraint, check constraint, Domain constraint, and Sequence value setting. Yes: Represent skipping post-schema migration, i.e., DTS does not migrate post-schema objects. No: Represent not skipping post-schema migration, i.e., DTS migrates post-schema objects. tip The configuration of whether to skip post-schema migration is only supported when the migration type includes Schema migration.
		Error retry time	If a migration task fails due to a disconnection between the source and target databases, the system immediately retries connecting to them multiple times within the specified time. Value range: 600 - 86400, in seconds (s). The default value is 600 seconds.
		Full migration rate	Set the full migration rate as needed. To prevent high migration rates from affecting the source database's performance, adjust the following parameters based on the read and write pressure of the source and target databases accordingly: Rows migrated per second: Enable and set the rows migrated per second as needed. The default value is 100,000, and the value range is 1 to 100,000. Data volume migrated per second: Enable and set the data volume migrated per second as needed. The default value is 25,000. The value range is 1 to 25,000, and the unit is KB. Migration task concurrency: Set the concurrency of migration tasks as needed. The default value is 8, the value range is 1-64. tip You can configure this parameter when selecting Migration type as Full migration. An excessively high migration rate may affect the performance of the source database. Adjust appropriately based on the read and write pressure of the source and target databases.

Last updated: 2026.09.02 16:10:47