Docs

Database Transmission Service

Volcano Engine Document Database for MongoDB as the source

Migrating to MongoDB connected over express network

This topic describes how to create a task for migrating the MongoDB connected over express network to the Volcano Engine MongoDB on the Database Transmission Service (DTS) console.

Prerequisites

A database and an account for Document Database for MongoDB have been created. For detailed operations, refer to Create instance and Create account.

Set the target information as needed, as detailed below:

When using VPN to perform data migration, the local network where the self-managed database is located has been connected to Volcano Engine via VPN gateway. For detailed steps, refer to Building a cloud VPC and on-premises multi-data-center network interconnection.

If data migration requires a dedicated line connection, you need to establish a dedicated line connection between a single cloud VPC and a single on-premises data center network. For more information, refer to Configuring direct connections.

The network connectivity and service availability of the source and destination have been confirmed.

Optional. Check each migration object in the source and target databases according to the instructions in the pre-check items. For more information, refer to Pre-check items (MongoDB).

Precautions

If incremental migration is not selected, do not write new data to the source instance during data migration to ensure data consistency.

To ensure real-time data consistency, it is recommended to select schema migration, full data migration, and incremental data migration.

Since full data migration performs concurrent insert operations, the collection in the target database generates fragments. Therefore, the storage space of the collection in the target database will be larger than that of the collection in the source database after the full migration is complete.

During the pre-check, if you ignore the alert and start the migration task, it may lead to data inconsistencies and pose business risks.

During migration, if there are CREATE TABLE or RENAME TABLE operations on the source end, select the entire database containing the table or the entire database where the table is renamed to when configuring the migration object. Otherwise, the newly created or renamed table will not migrate to the destination.

When creating a VPC data migration task, the system creates several network cards by default in your selected VPC, and the network cards are mounted by default to the subnet you selected. The network card name format is dts_shuttle_********.

The migration feature for cloud disk version MongoDB instances is currently in invitation-only testing. To use this feature, submit tickets to contact technical support to apply for usage.

Limits

Application limits

Type			Description
Application limits	Source end and destination restrictions	Bandwidth requirements: Adequate egress bandwidth is required on the source server. Failure to meet this prerequisite will lead to suboptimal migration performance. If the migration object is at the collection level and needs to be edited (e.g., collection name mapping), each migration task only supports migrating up to 1,000 collections. If this limit is exceeded, an error will appear after task submission. In this case, it is recommended to split the collections to be migrated and configure the task in batches. To perform incremental migration, complete the following configuration: Enable Oplog logs. Otherwise, an error will be reported during the pre-check phase, and the migration task can not start. It is recommended that the Oplog be retained for at least 7 days during full migration. To adjust the Oplog size, refer to Change the size of the Oplog. You can also clean the Oplog generated by the DTS source database after the full migration completes and the task runs. If the source is a MongoDB sharded cluster and oplog-based migration is used, the balancer must be closed before migration, and it must be ensured that no orphaned documents exist on any shard. For more information, refer to orphaned documents and how to clean orphaned documents in MongoDB (sharded cluster architecture).
Application limits		Other limits	It is recommended that the source and target Document Database for MongoDB be the same version, or migrate from a lower version to a higher one to ensure compatibility. If migrating from a higher version to a lower version, there may be database compatibility issues. Migrating data from admin and local databases is not supported. Transaction information is not retained, that is, transactions in the source database are converted into single records when migrated to the target database. Prior to executing data migration, assess the performance of both source and target databases. It is advised to perform the migration during off-peak business hours. Otherwise, during full migration, DTS consumes certain read/write resources on both source and target databases, which may lead to an increase in database load. The storage space of the target Document Database for MongoDB database must be greater than the storage space occupied by the source Document Database for MongoDB database. DTS uses ROUND(COLUMN,PRECISION) to read the migration precision of FLOAT and DOUBLE columns. The default migration precision is 38 digits for FLOAT and 308 digits for DOUBLE. Submit a ticket if requirements are not met.
Operation limits			Operation limits of the source database: During the schema and full migration phases, do not perform schema changes to databases or collections. Otherwise, the data migration task will fail. If only full data migration is performed, do not write new data to the source database, or the source and target data will be inconsistent.

Database account minimum permission policy

The account used for data migration must have the following permissions:

If the source database is a sharded cluster instance, you need to have the root account privileges of the source database.

tip

When incrementally migrating sharded cluster data and the incremental data acquisition method is ChangeStream, the source database account needs the ChangeStream permission.

If the source database is a replica set instance, the following permissions are required.

FIND, buildInfo, serverStatus, and listDatabases permissions of the object to be migrated from the source database.

The source database account must have read permissions for the database to be synchronized, the Config database, the admin database, and the local database.

The target database account must have read and write permissions for the target database, and read permissions for the local database.

On the Create migration task page, locate the Configure source and target database wizard, and configure the following parameters.

Type	Name	Description
None	Task name	Optional. Customize the task name. Naming rules are as follows: Must start with a Chinese character or letter. Must be 64 characters or less. It can contain only Chinese characters, letters, digits, underscores (_), and hyphens (-).
None		Link specification	Select the link specification for the migration task. Compact, Medium and Standard are supported. tip The link specification can not be modified after the task is successfully created. For details on the link specifications, refer to Product specifications.
Source database configuration	Select an existing data source	You can choose whether to use an existing instance as needed: Use existing instances: Click Select an existing data source, then in the Select data source control panel, choose the target region from the Region drop-down list, and select the target data source. Supports selecting data sources by data source name and ID. For methods to create data sources, refer to Data source management. Do not use existing instances: If not using an existing instance, you need to configure the following database instance information.
		Instance type	Select MongoDB.
		Access method	Select Volcano Engine Database for MongoDB.
		Region	Select the region where the target instance is located.
		Architecture	Select the architecture type of the target MongoDB instance as needed. currently supported options are replica set and sharded cluster.
		Incremental data acquisition method	When selecting Architecture as Sharded cluster, you can set the Incremental data acquisition method Select the incremental data acquisition method as needed. The following options are supported: ChangeStream: This method provides a way to obtain change events of specific types of modified data from a data set (database, collection, or pipeline). For example, insert, update, or delete events. If this acquisition method is selected, the instance version must be 3.6 or higher. Oplog: This acquisition method means directly obtaining the operation log of the MongoDB. For MongoDB sharded clusters, all sharding addresses must be entered after selecting this acquisition method. This method offers better synchronization performance compared to ChangeStream. None: After selecting None, incremental migration is not available. Only full migration can be selected.
		Cross-Volcano Engine account?	Select whether to transmit data cross-account as needed, with the following options: This account: Selecting this means data transmission occurs only within this account. Cross-account: Selecting this means data transmission occurs between two accounts. tip When transmitting data across accounts, log on to the Database Transmission Service (DTS) console through the account of the destination. Before cross-account data transmission, create and authorize an IAM role in the source root user. For detailed steps, refer to How to authorize users for cross-account data transmission.
		Cross-account ID	Enter the Volcano Engine account ID to which the source database belongs.
		Cross-account roles	Enter the IAM role authorized for DTS to access cross-account resources.
		MongoDB instance	Select the target instance of Document Database for MongoDB from the drop-down list. tip You can also search for the target instance by instance ID or name.
		Database account	Enter the database account of the target instance.
		Database password	Enter the password corresponding to the database account of the target instance.
		Test connection	Click Test connection to verify the connection to the instance. If the test connection fails, modify the configuration based on the failure prompt, and then click Test connection again to verify the connection. The details are as follows: Connection authentication failed. Check if the username and password are correct: When this error prompt appears, verify your username and password. Connection authentication failed, unable to create connection normally, contact customer service: When this error prompt appears, submit a ticket for technical support.
Target database configuration	Select an existing data source	You can choose whether to use an existing instance as needed: Use existing instances: Click Select an existing data source, then in the Select data source control panel, choose the target region from the Region drop-down list, and select the target data source. Supports selecting data sources by data source name and ID. For methods to create data sources, refer to Data source management. Do not use existing instances: If not using an existing instance, you need to configure the following database instance information.
		Instance type	Select MongoDB.
		Access method	Select MongoDB connected over express network.
		Architecture	Select the architecture type of the target MongoDB instance as needed, currently supporting Replica set, Sharded cluster, and Single node.
		Region	Select the region of the virtual private cloud.
		Address	Enter the host address and port of the target self-managed instance in the format of IP:Port. If multiple addresses exist, separate them with commas (,). Up to 5 addresses are supported. tip Currently, only IPv4 addresses are supported for access. IPv6 addresses are not yet supported.
		Account verification database	Enter the database name to which the account belongs.
		VPC	Select the target VPC from the drop-down list as follows: Direct connect: Select any VPC under the CEN when achieving data connectivity through a dedicated line CEN. VPN connection: Select the VPC bound to the VPN gateway when achieving data connectivity through VPN.
		Subnet	Select a subnet within the VPC from the drop-down list. tip When migrating a MongoDB instance connected over express network, you need to add subnet CIDR block access permissions in the allowlist or security group policy on the source end in advance. Click View subnet CIDR blocks to view and copy CIDR blocks.
		Database account	Enter the database account of the target instance.
		Database password	Enter the password corresponding to the database account of the target instance.
		Test connection	Click Test connection to verify the connection to the instance. If the test connection fails, modify the configuration based on the failure prompt, and then click Test connection again to verify the connection. The specific prompts are as follows: Connection authentication failed. Check the network: When this error prompt appears, confirm network connectivity. Connection authentication failed. Check the username and password: When this error prompt appears, verify your username and password. Connection authentication failed, unable to create connection normally, contact customer service: When this error prompt appears, submit a ticket to contact technical staff. Invalid IP address: When this error prompt appears, verify the accuracy of the IP address. The Express network requires the subnet to have * available IPs. The current number of available IPs in the subnet is insufficient, and the task can not be created. Check the subnet resources in the VPC: When this error prompt appears, adjust the number of available IPs in the VPC subnet to ensure enough IPs in the target VPC subnet.
None	Project	Optional. Select the affiliated project for the task from the drop-down list. By default, it is under default (default project). You can also click Create new project to create a new project. For more information, refer to Creating a new project. tip To grant sub-user privileges to access resources in a specific project, refer to Modifying project privileges. After a task is created successfully, if you need to change its affiliated project, you can move the resource to another project. For more information, refer to Moving resources to a project. A project is a virtual concept that includes a set of resources, users, and roles. A project allows for a unified view and management of a set of resources, and controls the permissions of users and roles within the project for those resources.
		Tag	(Optional) Click Add tag to add a tag to the instance. Tags can help you categorize cloud resources, control access to them, and allocate bills and costs. A tag is a key-value pair that identifies a cloud resource. You can use tags to categorize resources with the same characteristics from different dimensions, such as purpose or owner, making it easier to filter, aggregate, and manage your cloud resources. For more details about tags, refer to Tag overview. The rules for tag key-value pairs are as follows: Key Supports any language, characters, numbers, spaces, or symbols (_.:/=+-@). Case-sensitive, and the length is 1 to 128 characters. In UTF-8 encoded Unicode format. Keys cannot start with any form of volc:, such as Volc: or VOLC:. (Optional) Value Supports any language, character, digit, space, or symbol (_.:/=+-@). Case-sensitive, and the length is 0 to 256 characters. In UTF-8 encoded Unicode format. tip Support fuzzy search for selecting existing tag keys and tag values. A maximum of 20 tags can be associated with a task in a single operation. A maximum of 50 tags can be associated with a single task. For a single resource, tag keys must be unique, and each tag key can have only one value.
		Billing method	Currently, only Pay-as-you-go is supported. No selection is required. tip For detailed billing information, refer to Billing instructions.

Click Next step: configure migration object.

On the Configure migration object wizard page, configure the related parameter information as shown in the table below.

tip

If you do not need to start the migration task immediately, click Save in the lower-right corner of the page after configuring the migration object.

After saving, you will be redirected to the Migration task list page. The task status will be To be started, and you can configure the migration task at a suitable time. For configuration methods, refer to Configuring a migration task.

Name	Description
Migration type	Select the migration type based on business needs, including: If only full migration is required, select both Schema migration and Full migration. If online migration is needed, select Schema migration, Full migration, and Incremental migration simultaneously. tip If incremental migration is not selected, do not write new data to the source instance during data migration to ensure data consistency.
Migration object	Check the objects to migrate in the Migration object area. You can use Normal search and Global search to find target objects. At the same time, you can perform the following operations in the Selected object area: Rename databases or tables: Click the pencil icon next to the target database, table, or view to rename them. If you select entire databases in the Migration object area, you can only modify the database name in the Selected object area, and can not expand the database. Click Batch mapping to modify database or table names in bulk. For more information, refer to Database and table batch mapping. Filter and delete unnecessary targets: The search box in the Selected object area supports fuzzy search to batch delete unwanted databases, tables, and other targets.
Error retry time	If a migration task fails due to a disconnection between the source and target databases, the system immediately retries connecting to them multiple times within the specified time. Value range: 600 - 86400, in seconds (s). The default value is 600 seconds.
Full migration rate	Set the full migration rate as needed. To prevent high migration rates from affecting the source database's performance, adjust the following parameters based on the read and write pressure of the source and target databases accordingly: Rows migrated per second: Enable and set the rows migrated per second as needed. The default value is 100,000, and the value range is 1 to 100,000. Data volume migrated per second: Enable and set the data volume migrated per second as needed. The default value is 25,000. The value range is 1 to 25,000, and the unit is KB. Migration task concurrency: Set the concurrency of migration tasks as needed. The default value is 8, the value range is 1-64. tip You can configure this parameter when selecting Migration type as Full migration. An excessively high migration rate may affect the performance of the source database. Adjust appropriately based on the read and write pressure of the source and target databases. For more information, refer to Migration rate settings for MongoDB.

Last updated: 2026.09.02 16:10:47