Torch Object Storage
Torch Object Storage
Torch Object Storage
Document
What's New
Announcements
Billing
Billable items
Billing method
Getting Started
User Guide
Working with buckets
Basic configurations
Managing lifecycle
Back-to-origin rules
Event notification
Intelligent Tiering
Permissions
Domain management
Fault tolerance and disaster recovery
Cross-region replication
Same-region replication
Data redundancy
Image processing
ObjectSet
Working with objects
Managing folders
Monitoring and alerting
Metric monitoring
Access control
Bucket policies
Common configuration scenarios
Authorizing IAM users of the current account
Authorizing other accounts
Managing access via tools and SDKs
Managing public read access
Data security
Access points
Multi-region access points (MRAP)
Single-region access points (SRAP)
Converged access points (CAP)
DevMind
TOS accelerator
SenseFlow
TOS Agent
Data processing
Basic concepts
Image processing
Media processing
Document preview
Task processing
Data subscription
Flow control management
Quality of Service (QoS) policies
Hierarchical namespace (HNS)
User guide
Tool reference
Best practices
TOS Vectors
User guide
API reference
Vector bucket
Vector index
SDK reference
Python-Vectors
Vector bucket
Vector index
Java-Vectors
Vector bucket
Vector index
Node.js-Vectors
Vector bucket
Vector index
Tool Guide
tosutil
Common commands
Upload, download, and copy objects
Share objects
Auxiliary commands
Common examples
TOS Browser
FSX
FSX best practices
TOS PyTorch Connector
tosfsspec
API Reference
Signature Mechanism
The header contains a signature
Bucket
Basic Bucket Operations
Bucket Lifecycle
Access Monitor
Bucket ACL
Bucket CORS
Bucket Policy
Back-to-origin Rules
Bucket Replication
Versioning
Static Website
Event Notification
Custom Domain
Server-side encryption
Transfer Acceleration
Data Processing
Image processing
Media processing
Video transcoding
Audio transcoding
video to GIF
Document preview
File decompression
Log Storage
Retention Policy
Requester pays
TLS version
Object
Basic Object Operations
Object ACL
Fetch Object
Symbolic Link
WORM object
Multipart Upload
SDK Reference
Java
Pre-signing Mechanism
Bucket Interface
Basic Bucket Interface
Manage Bucket Configuration
Object Interface
Upload Object
Download Object
Upload and Download Speed Limit
Copy Object
List Object
Data Processing
Image Processing
Video Processing
Audio Processing
Log Configuration
Hierarchical Namespace
Bucket base interface
Introduction to Typical Scenes
Python
Data processing
Pre-signature mechanism
Bucket interface
Basic bucket interface
Manage bucket configuration
Object interface
Upload object
Download object
Copy object
List objects
Introduction to typical scenes
C
Object Interface
Upload object
C++
Pre-signature Mechanism
Bucket Interface
Basic Bucket Interface
Manage Bucket Configuration
Object Interface
Upload Object
Download Object
Copy Object
List Objects
Data Processing
Go
Object Interface
Upload Object
Download Object
Copy Object
List Objects
Manage Objects
Bucket Interface
Manage Bucket Configuration
Data Processing
Log Configuration
Introduction to Typical Scenes
Pre-signature Mechanism
Node.js
Pre-signature Mechanism
Bucket Interface
Basic Bucket Interface
Manage Bucket Configuration
Object Interface
Upload object
Download object
Copy object
List objects
Data Processing
Image Processing
Introduction to Typical Scenes
Browser.js
Pre-signature Mechanism
Upload Object
Download Object
Copy Object
List Objects
Manage Objects
Data Processing
.NET
Bucket Interface
Object Interface
Upload Object
Download Object
List Objects
Introduction to Typical Scenes
PHP
Bucket Interface
Object Interface
Download Object
Data Processing
Exception Handling
Android
Bucket Interface
Object Interface
Upload Object
Download Object
Exception Handling
iOS
Bucket Interface
Object Interface
Upload Object
Download Object
Harmony
Pre-signature Mechanism
Bucket Interface
Object Interface
Upload Object
Download Object
List Objects v2
Introduction to Typical Scenarios
Rust
Pre-signing mechanism
Bucket interface
Basic bucket operations
Lifecycle rules
Bucket ACLs
Cross-origin resource sharing
Bucket inventory
Back-to-origin rules
Bucket replication
Static website hosting
Event notification
Custom domain
Server-side encryption
Object interface
Basic operation
Access control
Error Codes
01-SERVICE_ERROR
02-AUTH
03-ACCESS_CONTROL
04-QOS
05-REQUEST
06-BUCKET
07-LIFECYCLE
08-ACL
09-CORS
10-POLICY
11-MIRROR
12-REPLICATION
13-WEBSITE
14-NOTIFICATION
15-ENCRYPT
16-CUSTOMDOMAIN
17-OBJECT
18-TAG
19-MULTIPART
20-STORAGE_CLASS
21-CALLBACK
22-DATA_PROCESS
23-BATCH
24-STORAGE_LENS
25-INVENTORY
26-DIRECTORY_BUCKET
FAQ
Best Practices
AIGC-Doubao LLMs
Accessing TOS with third-party tools
Granting temporary access
Data migration
Data processing
Checking data integrity
Performance optimization
- Documentation
Torch Object StorageSDK ReferencePythonBucket interfaceManage bucket configurationManage cross-region replication (Python SDK)
Copy page
Download PDFManage bucket configuration
Manage cross-region replication (Python SDK)
Copy pageDownload PDFManage cross-region replication (Python SDK)
TOS supports cross-region replication, allowing you to copy objects from one region to buckets in different regions. When cross-region replication rules are enabled, when you upload a new file in the source bucket, TOS automatically syncs the file to the destination bucket. This function is used to meet the requirements of off-site disaster recovery and data replication.
Set cross-regional replication rules
Precautions
- After cross-region rules are turned on, the corresponding files will be synchronized.
- Cross-region replication uses an asynchronous replication mechanism that can take anywhere from a few minutes to a few hours, depending on the size of your file.
- A single bucket can create up to 100 cross-region replication rules.
- Cross-region replication can only be enabled if the source and destination buckets have the same versioning state.
- To set cross-regional replication rules for buckets, you must be the bucket owner by default.
- When the CRR task is executed, when both StorageClassTypeand StorageClassInheritDirectiveTypeare set, the configuration of StorageClassTypeshall prevail.
- When only StorageClassInheritDirectiveTypeis set, the configuration value is consistent with the source object when it is Storage_Class_ID_Source_Object, and the configuration item value is consistent with the target bucket when it is Storage_Class_ID_Destination_Bucket.
- When neither StorageClassTypenor StorageClassInheritDirectiveTypeis set, the default policy is used to match the storage type of the target bucket.
Sample code
The following code sets the cross-region replication rules for bucket-test.
import os
import tos
from tos import StatusType, StorageClassInheritDirectiveType, StorageClassType
from tos.models2 import ReplicationRule, Destination
# Retrieve AKSK information from environment variables.
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
# Fill in "your endpoint" and "your region" with the corresponding Endpoint of the region where your Bucket is located. Taking the example of North China 2 (Beijing), fill in "your endpoint" with tos-cn-beijing.volces.com and "your region" with cn-beijing.
endpoint = "your endpoint"
region = "your region"
dest_region = "your crr dest region"
bucket_name = "bucket-test"
dest_bucket_name = "bucket-crr"
try:
# Create TosClientV2 object to perform operations on buckets and objects
client = tos.TosClientV2(ak, sk, endpoint, region)
# Set cross-region public rules
rule = ReplicationRule(
# Rule ID
id='1',
# Specify whether to enable cross-region replication rule
status=StatusType.Status_Enable,
# Specify the destination where the data should be replicated to
destination=Destination(
# Specify the target Bucket where the data should be copied to
bucket=dest_bucket_name,
# Specify the Region where the target Bucket is located
location=dest_region),
historical_object_replication=StatusType.Status_Enable
)
# Specify the prefix of the matching objects for cross-region replication. Only objects that match this prefix will be copied to the destination bucket.
# prefix_set = ['prefix1', 'prefix2']
# Set up the cross-region replication rule.
# replica_config = ReplicationRule(
# # Rule ID
# id='1',
# # Specify whether to enable the cross-region replication rule (Status_Enable) or disable it (Disabled)
# status=StatusType.Status_Enable,
# # Specify the destination where the data should be replicated to
# destination=Destination(
# # Specify the destination bucket where the data should be replicated to
# bucket=dest_bucket_name,
# # Specify the region where the destination bucket is located
# location=dest_region,
# # Specify the storage class
# storage_class=StorageClassType.Storage_Class_Ia,
# # Specify the storage class of the replicated objects, inherit the storage class of the source object (Storage_Class_ID_Source_Object) or the destination bucket (Storage_Class_ID_Destination_Bucket)
# storage_class_inherit_directive=StorageClassInheritDirectiveType.Storage_Class_ID_Source_Object
# ),
# # Specify the object prefix for the matching rule
# prefix_set=prefix_set,
# # Specify whether to enable replication of historical objects
# historical_object_replication=StatusType.Status_Enable,
# )
# Set up the cross-region replication rule
# When configuring the cross-region replication rule, you must create the corresponding role in the access configuration and grant access permissions to TOS resources
# The role_name is the name of the role created above.
client.put_bucket_replication(bucket_name, 'your role', [rule])
except tos.exceptions.TosClientError as e:
# Operation failed, catch client error, usually caused by invalid request parameters or network issues
print('fail with client error, message:{}, cause: {}'.format(e.message, e.cause))
except tos.exceptions.TosServerError as e:
# Operation failed, catch server error, detailed error information can be obtained from the response
print('fail with server error, code: {}'.format(e.code))
# request id can be used to locate specific issues, it is strongly recommended to save it in the log.
print('error with request id: {}'.format(e.request_id))
print('error with message: {}'.format(e.message))
print('error with http code: {}'.format(e.status_code))
print('error with ec: {}'.format(e.ec))
print('error with request url: {}'.format(e.request_url))
except Exception as e:
print('fail with unknown error: {}'.format(e))
View cross-region replication rules
warning
To obtain cross-regional replication rules for buckets, you must be the bucket owner by default.
Sample code
The following code is used to obtain cross-region replication rules for bucket-test.
import os
import tos
# Retrieve AKSK information from environment variables.
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
# Fill in "your endpoint" and "your region" with the corresponding Endpoint of the region where your Bucket is located. Taking the example of North China 2 (Beijing), fill in "your endpoint" with tos-cn-beijing.volces.com and "your region" with cn-beijing.
endpoint = "your endpoint"
region = "your region"
bucket_name = "bucket-test"
try:
# Create a TosClientV2 object to perform operations on buckets and objects
client = tos.TosClientV2(ak, sk, endpoint, region)
# Retrieve the replication rules for all objects in the bucket (when rule_id is not specified)
out = client.get_bucket_replication(bucket_name)
# Retrieve the replication rule with the specified rule_id
# out = client.get_bucket_replication(bucket_name, rule_id='your rule id')
# Output all the replication rules configured for the bucket
for rule in out.rules:
# Output the current rule ID's cross-region replication rule configuration information.
print('rule id', rule.id)
print('prefix set', rule.prefix_set)
print('status', rule.status)
print('historical object replication', rule.historical_object_replication)
print('location', rule.destination.location)
print('targe bucket', rule.destination.bucket)
print('storage_class', rule.destination.storage_class)
print('storage class inherit directive', rule.destination.storage_class_inherit_directive)
except tos.exceptions.TosClientError as e:
# Operation failed, catch client error, usually caused by invalid request parameters or network issues
print('fail with client error, message:{}, cause: {}'.format(e.message, e.cause))
except tos.exceptions.TosServerError as e:
# Operation failed, catch server error, detailed error information can be obtained from the response
print('fail with server error, code: {}'.format(e.code))
# request id can be used to locate specific issues, it is strongly recommended to save it in the log.
print('error with request id: {}'.format(e.request_id))
print('error with message: {}'.format(e.message))
print('error with http code: {}'.format(e.status_code))
print('error with ec: {}'.format(e.ec))
print('error with request url: {}'.format(e.request_url))
except Exception as e:
print('fail with unknown error: {}'.format(e))
View cross-region replication progress
warning
- To get the cross-regional replication progress of a bucket, you must be the bucket owner by default.
- Historical data replication progress is expressed as a percentage and is only valid for storage spaces where historical data replication is enabled.
- The new write data replication progress is represented by the point in time of the new write data.
Sample code
The following code is used to get the cross-region replication progress for bucket-test.
import os
import tos
# Retrieve AKSK information from environment variables.
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
# Fill in "your endpoint" and "your region" with the corresponding Endpoint of the region where your Bucket is located. Taking the example of North China 2 (Beijing), fill in "your endpoint" with tos-cn-beijing.volces.com and "your region" with cn-beijing.
endpoint = "your endpoint"
region = "your region"
bucket_name = "bucket-test"
try:
# Create a TosClientV2 object to perform operations on buckets and objects
client = tos.TosClientV2(ak, sk, endpoint, region)
# Return the replication progress of all objects without specifying a rule_id
out = client.get_bucket_replication(bucket_name)
# Query the replication progress of a specific rule_id by specifying rule_id='your rule id'
# out = client.get_bucket_replication(bucket_name, rule_id='your rule id')
# Output the replication progress of all objects currently configured for the bucket
for rule in out.rules:
# Output the cross-region replication progress for the current rule id.
print('rule id', rule.id)
print('new object progress', rule.progress.new_object)
print('historical object progress', rule.progress.historical_object)
except tos.exceptions.TosClientError as e:
# Operation failed, catch client error, usually caused by invalid request parameters or network issues
print('fail with client error, message:{}, cause: {}'.format(e.message, e.cause))
except tos.exceptions.TosServerError as e:
# Operation failed, catch server error, detailed error information can be obtained from the response
print('fail with server error, code: {}'.format(e.code))
# request id can be used to locate specific issues, it is strongly recommended to save it in the log.
print('error with request id: {}'.format(e.request_id))
print('error with message: {}'.format(e.message))
print('error with http code: {}'.format(e.status_code))
print('error with ec: {}'.format(e.ec))
print('error with request url: {}'.format(e.request_url))
except Exception as e:
print('fail with unknown error: {}'.format(e))
Delete cross-region replication rules
warning
Remove the cross-region replication rule for buckets. By default, you must be the bucket owner.
Sample code
The following code removes the cross-region replication rule for bucket-test.
import os
import tos
# Retrieve AKSK information from environment variables.
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
# Fill in "your endpoint" and "your region" with the corresponding Endpoint of the region where your Bucket is located. Taking the example of North China 2 (Beijing), fill in "your endpoint" with tos-cn-beijing.volces.com and "your region" with cn-beijing.
endpoint = "your endpoint"
region = "your region"
bucket_name = "bucket-test"
try:
# Create TosClientV2 object to perform operations on buckets and objects
client = tos.TosClientV2(ak, sk, endpoint, region)
# Delete cross-region replication rule
client.delete_bucket_replication(bucket_name)
except tos.exceptions.TosClientError as e:
# Operation failed, catch client error, usually caused by invalid request parameters or network issues
print('fail with client error, message:{}, cause: {}'.format(e.message, e.cause))
except tos.exceptions.TosServerError as e:
# Operation failed, catch server error, detailed error information can be obtained from the response
print('fail with server error, code: {}'.format(e.code))
# request id can be used to locate specific issues, it is strongly recommended to save it in the log.
print('error with request id: {}'.format(e.request_id))
print('error with message: {}'.format(e.message))
print('error with http code: {}'.format(e.status_code))
print('error with ec: {}'.format(e.ec))
print('error with request url: {}'.format(e.request_url))
except Exception as e:
print('fail with unknown error: {}'.format(e))
Last updated: 2025.09.17 18:22:13
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
