Docs

Database Transmission Service

MySQL connected over public network as the source

Migrating to MySQL connected over express network

This topic describes how to create a task to migrate a public network MySQL to a MySQL connected over express network in the Database Transmission Service (DTS) console.

Prerequisites

You have set up MySQL in a public network environment and created a database account.

Set the target information as needed, as detailed below:

When using VPN to perform data migration, the local network where the self-managed database is located has been connected to Volcano Engine via VPN gateway. For detailed steps, refer to Building a cloud VPC and on-premises multi-data-center network interconnection.

If data migration requires a dedicated line connection, you need to establish a dedicated line connection between a single cloud VPC and a single on-premises data center network. For more information, refer to Configuring direct connections.

The network connectivity and service availability of the source and destination have been confirmed.

Optional. Check each migration object in the source and target databases according to the instructions in the pre-check items. For details, refer to Pre-check items (RDS for MySQL).

Precautions

When the source is self-managed MySQL, pay attention to the following information:

If the source end switches to the primary during migration, the migration task will fail.

If the source end performs a restart operation during migration, the migration task will be temporarily interrupted and automatically attempted to recover. If the task does not recover for a long time, submit a ticket for technical support.

Since the delay time of Database Transmission Service (DTS) is derived by comparing the timestamp of the last data migrated to the destination with the current timestamp, the delay information may be inaccurate if the source end has not performed DML operations for a long time. If the delay time displayed for the task is too long, you can execute a DML operation in the source end to update the delay information.

tip

If the migration object is the entire database, you can create a heartbeat table that is updated or written with data every second.

When the target end uses a high-privilege account, DTS automatically creates the database at the destination. If the database name to be migrated does not comply with the target database naming conventions, you need to create the database at the destination before the migration task.

During user migration, if some source database accounts do not support migration, ensure that business operations are not affected by the absence of these accounts.

During migration, if there are CREATE TABLE or RENAME TABLE operations on the source end, select the entire database containing the table or the entire database where the table is renamed to when configuring the migration object. Otherwise, the newly created or renamed table will not migrate to the destination.

When the migration object is not the entire database, OnlineDDL operations are not supported. Meanwhile, incremental migration does not automatically migrate newly added objects and their changes.

The migration object granularity can be either database or table. If tables are selected as the migration objects, other objects such as views, triggers, functions, or stored procedures will not be migrated to the target end.

To ensure data migration performance and task stability, the source TRIGGER and EVENT will migrate only after the incremental migration completes.

During the pre-check, if you ignore the alert and start the migration task, it may lead to data inconsistencies and pose business risks.

When creating a data migration task via express network, the system creates several network cards by default in your selected VPC, and the network cards are mounted by default to the subnet you selected. The network card name format is dts_shuttle_********.

During data migration, if duplicate primary key values or unique key values are encountered in the target database, DTS will not retain the conflicting data at the destination. Instead, the data from the source will overwrite the data at the destination. When a conflict overwrite occurs, if there is a table with the same name in the target database, it may cause data inconsistency issues as detailed below. It is recommended to use table name mapping or delete or rename the table with the same name in the target database to avoid data inconsistency and reduce business risks.

When the table structure is consistent, the migrated data overwrites records in the target table with the same primary key or unique key values as in the source database.

When the table structure is inconsistent, initialization of data may fail, and only partial column data can be migrated.

Do not perform frequent DDL operations on the same table, as repeated DDL operations may cause DTS metadata update failures and risk transmission interruptions. Avoid this by spacing DDL operations at least 1–3 seconds apart in the console.

Limits

Type	Description
Source limits	Database version: MySQL 5.5, 5.6, 5.7, and 8.0 instances are supported. tip For MySQL 5.5 instances, only schema and data migration are supported. User migration is not available. When the instance version of the source database is greater than or equal to 5.6, the parameter gtid_mode is set to ON. tip When the MySQL version is earlier than 8.0 and the parameter gtid_mode is disabled, foreign key cascading deletion may fail during data migration, resulting in data differences. Bandwidth requirements: Adequate egress bandwidth is required on the source database server. Failure to meet this prerequisite will lead to suboptimal migration performance. Data migration supports storage engines including InnoDB, MyISAM, and RocksDB. During data migration, if the source database is not using the InnoDB storage engine, the system automatically converts it to InnoDB. MariaDB database engine is not supported currently. Partitioned tables are not supported when converting the MyISAM storage engine to the InnoDB storage engine. Tables to be migrated must have a primary key or a unique non-null constraint, and their fields must be unique. Otherwise, data inconsistency may occur. If a table to be migrated is an extra-large table and lacks a primary key or unique non-null constraint, the data migration process may stall when the network is unstable. To perform incremental migration, Binlogs must meet the following requirements: The Binlog feature must be enabled, and the parameter binlog_format set to row, binlog_row_image set to full. Otherwise, pre-check will report an error, and data migration tasks can not start successfully. Binlogs must be retained for at least 24 hours, and a retention period of more than seven days is recommended. Otherwise, the DTS task may fail because it can not obtain logs, potentially leading to data loss. Tables with foreign key dependencies must be migrated together, or data migration will fail. It is recommended that the number of tables in a single task should not exceed 20,000, and the number of databases should not exceed 1,000. When a large number of databases and tables need to be migrated, it is recommended to split them into multiple tasks. The source database does not support configuring multi-node proxy addresses. During incremental data migration, DTS requires establishing a stable long connection with specific database instances. The load balancing or routing mechanism of multi-node proxy may cause connection drift or log parsing failure. It is recommended to configure a single node connection address directly or use a database gateway (DG) for access.
Target limits	Database version: MySQL 5.5, 5.6, 5.7, and 8.0 instances are supported. The migration account privileges of the target database will affect the use of views, stored procedures, and functions. The specific limits are as follows: When the migration account of the target database has SET_USER_ID permission, the limits for views, stored procedures, and functions are as follows: Views, stored procedures, and functions will retain the original definer and sql_security. The views, stored procedures, and functions can be used normally only after the account migration task is completed or the corresponding account is manually created in the target database. When the migration account of the target database does not have or support SET_USER_ID permission, after migrating views, stored procedures, and functions, the definer will be replaced by the migration account, and sql_security will be set to invoker. tip Only MySQL 8.0 supports setting SET_USER_ID permission.
DDL limitations	Operation limits General rules DDL operations are prohibited during initialization: Do not execute DDL during link creation, schema migration, or full data migration—otherwise, the task will fail. When business writes occur on the destination, executing DDL on the source end may cause data loss. For example: in incremental migration tasks, once the business traffic switches completely (i.e., writes have migrated to the destination), executing DDL on the source end is strictly forbidden, as it may result in severe consequences such as data loss or structural inconsistency on the destination. At this stage, any DDL operations must be performed on the destination. The source is only for migration backtracking or read-only queries. tip When using online DDL tools such as gh-ost or pt-online-schema-change, temporary tables are created during schema changes. Data written to the target database during operations like RENAME or DROP on these temporary tables may be lost. OnlineDDL support scope: Do not write data to the target database during DDL migration, as it may result in data loss. DTS supports migrating the following temporary tables (ensure the target database has write permissions): gh-ost: original table name_ghc, original table name_gho, original table name_del, original table name_del, tp_{change ID}_ogt_{original table name}, tp_{change ID}_g_{original table name} tp_{change ID}_del_{original table name}, tp_{change ID}_d_{original table name}. pt-osc: original table name _new, original table name _old. Table mapping (renaming) is mutually exclusive with Online DDL temporary table migration, only one can be enabled at a time. Even lightweight operations, such as adding a column to a small table, may cause DDL migration to fail if the temporary table exists for too short a duration. It is recommended to perform all such changes uniformly through DBW tickets. For detailed steps, refer to lock-free schema change tickets or lock-free data change tickets. Avoid frequent DDL operations: Repeatedly applying DDL to the same table may cause DTS metadata updates to fail, risking replication interruption. You can avoid this issue by: When performing DDL operations via the console, allow a minimum interval of 1–3 seconds between consecutive operations. Prefer using DBW lock-free schema change tickets to execute DDL operations. For detailed steps, refer to lock-free schema change tickets or lock-free data change tickets. Cross-database DDL restrictions: If a database is not included in the Migration object, DDL affecting migrated databases (such as cross-database views or foreign keys) cannot be executed on it; otherwise, the Migration task will fail. Special limitations Table schema changes must be forward-compatible and limited to the following safe DDL operations:CREATE TABLE, ADD COLUMN, MODIFY COLUMN (only to increase column length), CREATE INDEX, DROP INDEX. Prohibited operations: Modifying column data types, dropping columns, or reducing column lengths are not allowed, as they may cause data inconsistency or migration interruption.
Other limits	It is recommended that the source and target MySQL databases be the same version, or migrate from a lower version to a higher one to ensure compatibility. If you migrate from a higher version to a lower version, there may be database compatibility or data loss issues. Prior to executing data migration, assess the performance of both source and target databases. It is also advised to perform the migration during off-peak business hours. Otherwise, during full data migration, DTS consumes certain read/write resources on both source and target databases, which may lead to an increase in database load. During migration, the target database may generate a large number of Binlog files, causing the target database disk usage to be much higher than that of the original instance. It is recommended to adjust the Binlog retention policy in the target database before migration or synchronization to prevent excessive storage usage by Binlog files. For data migration tasks with database or table name mapping, the mapping may not take effect after the stored procedure and function are migrated. If the source database is MySQL 8.0, when configuring migration tasks, you must manually grant the relevant data migration permissions directly to the following accounts: The accounts for the source or target database are granted relevant permissions through the Role. The accounts that need to perform User migration are granted relevant permissions through the Role. Only migration of tables, views, stored procedures, functions, triggers, and events is supported.

Supported SQL operations

The supported SQL operations for incremental migration are shown in the following table.

Operation type	SQL statement
DML	INSERT, UPDATE, DELETE
DDL	ALTER TABLE, ALTER VIEW CREATE FUNCTION, CREATE INDEX, CREATE PROCEDURE, CREATE TABLE, CREATE VIEW DROP INDEX, DROP TABLE, DROP FUNCTION, DROP PROCEDURE RENAME TABLE TRUNCATE TABLE

Database account minimum permission policy

Database	Schema migration	Full migration	Incremental migration
Source	SELECT permission	SELECT permission	REPLICATION SLAVE, REPLICATION CLIENT, and SELECT permissions.
Target	Read/Write permissions	Read/Write permissions	Read/Write permissions

On the Create migration task page, locate the Configure source and target database wizard, and configure the following parameters.

Type	Name	Description
Basic information	Task name	Optional. Customize the task name. The length must be within 256 characters. tip If not set, the system generates a task named with dts- by default.
Basic information		Link specification	Select the link specification for the migration task. Compact, Medium and Standard are supported. tip The link specification can not be modified after the task is successfully created. For details on the link specifications, refer to Product specifications.
Source database configuration	Select an existing data source	You can choose whether to use an existing instance as needed: Use existing instances: Click Select an existing data source, then in the Select data source control panel, choose the target region from the Region drop-down list, and select the target data source. Supports selecting data sources by data source name and ID. For methods to create data sources, refer to Data source management. Do not use existing instances: If not using an existing instance, you need to configure the following database instance information.
		Instance type	Select MySQL.
		Access method	Select MySQL connected over public network. tip When migrating a MySQL connected over public network instance, you need to add DTS IP segment access permissions in the allowlist or security group policies of the source end in advance. Click View DTS CIDR blocks to view and copy the CIDR block.
		Region	Select the region where the target instance is located.
		Host address	Enter the host address of the target database's self-managed instance.
		Port	Enter the port number of the target self-managed instance.
		Database account	Enter the database account of the target instance.
		Database password	Enter the password corresponding to the database account of the target instance.
		Connection method	Select an Unencrypted or SSL-secured connection as needed to establish a more secure and reliable migration link. tip When you select an SSL-secured connection, you must first enable the SSL encryption feature for MySQL connected over public network. When the migration task chooses unencrypted connection and runs normally, enabling the SSL encryption feature for MySQL over public network instance may cause unencrypted tasks to fail.
		SSL encryption certificate	Click Upload SSL encryption certificate to upload the CA certificate. When the Connection method is set to SSL-secured connection, this configuration is supported.
		Test connection	Click Test connection to verify the connection to the instance. If the test connection fails, modify the configuration based on the failure prompt, and then click Test connection again to verify the connection. The specific prompts are as follows: Connection authentication failed. Check the network: When this error prompt appears, confirm network connectivity. Connection authentication failed. Check the username and password: When this error prompt appears, verify your username and password. Invalid IP address: When this error prompt appears, verify the accuracy of the IP address.
Target database configuration	Select an existing data source	You can choose whether to use an existing instance as needed: Use existing instances: Click Select an existing data source, then in the Select data source control panel, choose the target region from the Region drop-down list, and select the target data source. Supports selecting data sources by data source name and ID. For methods to create data sources, refer to Data source management. Do not use existing instances: If not using an existing instance, you need to configure the following database instance information.
		Instance type	Select MySQL.
		Access method	Select MySQL connected over express network.
		Region	Select the region of the virtual private cloud.
		Host address	Enter the host address of the MySQL instance connected over express network. tip Currently, only IPv4 addresses are supported for access. IPv6 addresses are not yet supported.
		Port	Enter the port of the MySQL instance connected over express network.
		VPC	Select the target VPC from the drop-down list as follows: Direct connect: Select any VPC under the CEN when achieving data connectivity through a dedicated line CEN. VPN connection: Select the VPC bound to the VPN gateway when achieving data connectivity through VPN.
		Subnet	Select a subnet within the VPC from the drop-down list. tip When migrating a MySQL instance connected over express network, you need to add subnet CIDR block access permissions to the allowlist or security group policy on the source end in advance. Click View subnet CIDR blocks to view and copy CIDR blocks.
		Database account	Enter the database account of the target instance.
		Database password	Enter the password corresponding to the database account of the target instance.
		Test connection	Click Test connection to verify the connection to the instance. If the test connection fails, modify the configuration based on the failure prompt, and then click Test connection again to verify the connection. The specific prompts are as follows: Connection authentication failed. Check the network: When this error prompt appears, confirm network connectivity. Connection authentication failed. Check the username and password: When this error prompt appears, verify your username and password. Connection authentication failed, unable to create connection normally, contact customer service: When this error prompt appears, submit a ticket to contact technical staff. Invalid IP address: When this error prompt appears, verify the accuracy of the IP address. The Express network requires the subnet to have * available IPs. The current number of available IPs in the subnet is insufficient, and the task can not be created. Check the subnet resources in the VPC: When this error prompt appears, adjust the number of available IPs in the VPC subnet to ensure enough IPs in the target VPC subnet.
Project and billing	Project	Optional. Select the affiliated project for the task from the drop-down list. By default, it is under default (default project). You can also click Create new project to create a new project. For more information, refer to Creating a new project. tip To grant sub-user privileges to access resources in a specific project, refer to Modifying project privileges. After a task is created successfully, if you need to change its affiliated project, you can move the resource to another project. For more information, refer to Moving resources to a project. A project is a virtual concept that includes a set of resources, users, and roles. A project allows for a unified view and management of a set of resources, and controls the permissions of users and roles within the project for those resources.
		Tag	(Optional) Click Add tag to add a tag to the instance. Tags can help you categorize cloud resources, control access to them, and allocate bills and costs. A tag is a key-value pair that identifies a cloud resource. You can use tags to categorize resources with the same characteristics from different dimensions, such as purpose or owner, making it easier to filter, aggregate, and manage your cloud resources. For more details about tags, refer to Tag overview. The rules for tag key-value pairs are as follows: Key Supports any language, characters, numbers, spaces, or symbols (_.:/=+-@). Case-sensitive, and the length is 1 to 128 characters. In UTF-8 encoded Unicode format. Keys cannot start with any form of volc:, such as Volc: or VOLC:. (Optional) Value Supports any language, character, digit, space, or symbol (_.:/=+-@). Case-sensitive, and the length is 0 to 256 characters. In UTF-8 encoded Unicode format. tip Support fuzzy search for selecting existing tag keys and tag values. A maximum of 20 tags can be associated with a task in a single operation. A maximum of 50 tags can be associated with a single task. For a single resource, tag keys must be unique, and each tag key can have only one value.
		Billing method	Currently, only Pay-as-you-go is supported. No selection is required. tip For detailed billing information, refer to Billing instructions.

Click Next step: configure migration object.

On the Configure migration object wizard page, configure the related parameter information as shown in the table below.

tip

If you do not need to start the migration task immediately, click Save in the lower-right corner of the page after configuring the migration object.

After saving, you will be redirected to the Migration task list page. The task status will be To be started, and you can configure the migration task at a suitable time. For configuration methods, refer to Configuring a migration task.

Name	Description
Migration type	Select the migration type as needed. Currently supported types are User migration, Schema migration, Full migration, and Incremental migration. including: If only full migration is required, select both Schema migration and Full migration. If online migration is needed, select Schema migration, Full migration, and Incremental migration simultaneously. If specified non-system accounts and their associated permissions need to be migrated from the source database to the target database, select User migration. tip When only Full migration is selected, after successfully creating the task, you can check the specific ending GtidSet (inclusive) for the full migration on the Task details tab. This indicates that all data before this GtidSet has been migrated to the target database, and any changes in the source database after this GtidSet will not be migrated. When Incremental migration is not selected, do not write new data to the source database instance during data migration to ensure data consistency. If User migration is needed, user and permission migration must be handled separately. If some source database accounts do not support migration, ensure that business operations are not affected by the absence of these accounts.
Migration user	Select the target accounts as needed, and set a new password in the New password (Optional) column. When migrating multiple users, click Batch change passwords to change all their passwords at once. Otherwise, the target account passwords will be the same as the source passwords. tip This configuration is supported when Migration type is set to User migration.
Migration starting point	Set to pull Binlog logs from a specified position or timestamp. The values are as follows: System default: The default is the task start time. Specify GTID: Enter the specified GTID in the input box, formatted as source_id:transaction_id. Set GTID_EXECUTED: Enter the specified GTID_EXECUTED in the input box, formatted as source_id:transaction_id. If multiple exist, separate them with commas (,). For how to obtain the instance's GTID_EXECUTED, refer to How to obtain the GTID_EXECUTED of the instance. Specify timestamp: Click the input box to select a date, click Select time to set the time point, then click OK. tip You can configure this parameter when selecting Migration type as Incremental migration. When Migration type selects both Full migration and Incremental migration, migration defaults to the current time point. GTID and GTID_EXECUTED must be legal values from the source end. Otherwise, pre-check reports an error. When selecting Specify timestamp, the time you set must be later than the earliest Binlog time on the source end. If there is no corresponding time Binlog, the closest Binlog starting point will be selected.
Whether to enable foreign key check?	Select whether to enable foreign key check as needed. When enabled, the tool supports migrating cascading updates and deletes of foreign keys from the source, ensuring data integrity and consistency throughout the migration.
Migration object	Check the objects to migrate in the Migration object area. You can use Normal search and Global search to find target objects. At the same time, you can perform the following operations in the Selected object area: Rename databases or tables: Click the pencil icon next to the target database, table, or view to rename them. If you select entire databases in the Migration object area, you can only modify the database name in the Selected object area, and can not expand the database. Click Batch mapping to modify database or table names in bulk. For more information, refer to Database and table batch mapping. Filter and delete unnecessary targets: The search box in the Selected object area supports fuzzy search to batch delete unwanted databases, tables, and other targets. Filter data in tables: Check Filter data enables filtering of data within the table. tip Regular expression search for target objects is supported in the Migration object or Selected object areas. When the migration object is not the entire database, OnlineDDL operations are not supported. Meanwhile, incremental migration does not automatically migrate newly added objects and their changes. The migration object granularity can be either database or table. If tables are selected as the migration objects, other objects such as views, triggers, functions, or stored procedures will not be migrated to the target database. To ensure data migration performance and task stability, the source TRIGGER and EVENT will migrate only after the incremental migration completes.
Filter condition	Optional. Filtering data during a full data scan improves migration performance. Click the pen icon in the Filter condition column for the target database, table, or view, set the filter condition, then click Verification statement to validate the condition. tip This feature is currently in beta testing. To use it, submit a ticket for technical support. This field can be configured only if Filter data is selected for the Migration object's database, table, or view. Filter condition currently supported =, !=, <, >, <=, >=, in, is null, and , or.
whether to pre-migrate trigger	Select whether to pre-migrate the migrate trigger as needed. The following options are supported: No: DTS migrates triggers by default after incremental migration ends. Yes: DTS migrates triggers before starting incremental migration and adds control statements in the target database triggers to avoid DTS triggering them. tip This feature is released in allowlist. To use it, submit tickets to contact technical support. The following conditions must be met to configure this parameter. Migration types include schema migration and incremental migration. Migration objects include trigger.
Trigger migration method	Select pre-migrate trigger migration methods as needed, supporting the following options: Automatic migration: DTS automatically migrates triggers before incremental migration and adds control statements in the target database's triggers to prevent DTS from triggering them. Manually trigger migration: DTS requires manual migration of triggers before incremental migration, and adding control statements to triggers in the target database to prevent DTS from triggering triggers in the target database. tip Configuring this parameter is supported only when whether to pre-migrate triggers is set to yes.
Enable transaction merging?	Select whether to enable transaction merging during data migration. Enabling transaction merging optimizes migration tasks by combining multiple individual transactions, improving overall efficiency and performance while ensuring data consistency. When data migration occurs frequently with large volumes, enabling transaction merging enhances the migration efficiency. No: Disable transaction merging. Yes: Enable transaction merging.
Migration preferences	Select a migration preference policy as needed. The following options are supported: Performance first: It can increase the speed of existing data migration but may also lead to higher disk fragmentation and increased system load. It is recommended to reserve an additional 50% of storage space on the target end. Performance first applies to scenarios with high requirements for existing data migration speed, such as financial transactions, fast data collection, and real-time monitoring. Space first: The test space amplification rate is generally around 5%, which is suitable for scenarios with limited storage resources and relatively low requirements for data migration timeliness, such as data archiving, historical data backup, and log file synchronization.
Error retry time	If a migration task fails due to a disconnection between the source and target ends, the system immediately retries connecting to them multiple times within the specified time. Value range: 600 - 86,400, in seconds (s). The default value is 600 seconds.
Full migration rate	Set the full migration rate as needed. To prevent high migration rates from affecting the source database's performance, adjust the following parameters based on the read and write pressure of the source and target databases accordingly: Rows migrated per second: Enable and set the rows migrated per second as needed. The default value is 100,000, and the value range is 1 to 100,000. Data volume migrated per second: Enable and set the data volume migrated per second as needed. The default value is 25,000. The value range is 1 to 25,000, and the unit is KB. Migration task concurrency: Set the concurrency of migration tasks as needed. The default value is 8, the value range is 1-64. tip You can configure this parameter when selecting Migration type as Full migration. An excessively high migration rate may affect the performance of the source database. Adjust appropriately based on the read and write pressure of the source and target databases.

Last updated: 2026.07.31 11:38:46