Docs

Torch Object Storage

Best Practices

Compatibility with the Amazon S3 protocol

This topic introduces a solution that allows you to effortlessly transition from any S3-compatible storage to TOS. It also covers related considerations and describes TOS-compatible S3 APIs.

Background information

TOS is compatible with the Amazon Simple Storage Service (S3) protocol. Once data is migrated from any S3-compatible storage solution to TOS, applications that previously accessed that storage solution can seamlessly switch to TOS with minimal adjustments.

Constraints and limitations

As TOS is compatible with the S3 protocol, you can use Amazon Web Services (AWS) SDKs for S3 or other S3-compatible tools to create buckets, upload objects, and perform other operations.

The bandwidth, QPS, and other limits must comply with the TOS rules. For more information, see Constraints and limitations.

Transitioning to TOS

Before transitioning, you must migrate any existing data in S3-compatible storage to TOS.

Step 1: Migrate data to TOS

You can use Volcano Engine Data Migration Service to migrate data from any S3-compatible storage to TOS. For more information, see Data Migration Service User Guide.

Step 2: Access TOS via the S3 protocol

After migrating your data to TOS, you can access it using AWS SDKs or other S3-compatible tools, such as the AWS CLI, S3 Browser, or Postman. Note the following when accessing TOS via the S3 protocol:

Selecting an endpoint

TOS provides specific endpoints for the S3 protocol (S3 endpoints). You must use the dedicated endpoint to access TOS via the S3 protocol.

warning

Unlike AWS and other cloud service providers, which use the same endpoint for private and public networks, TOS uses different endpoints for each. For example, the S3 endpoints for accessing TOS resources in China (Beijing) from the private and public networks are tos-s3-cn-beijing.ivolces.com and tos-s3-cn-beijing.volces.com, respectively.

For more information on TOS endpoints, see Regions and endpoints.

TOS supports AWS Signature Version 4 but not Version 2.

Differences in S3 and TOS endpoints

While AWS and other cloud service providers use a global endpoint (s3.amazonaws.com) to access buckets in any region, TOS uses regional endpoints. Therefore, a TOS bucket must be accessed using the appropriate regional endpoint. Examples:

Bucket
Region
Endpoint
bucket-test1
China (Beijing)
tos-s3-cn-beijing.volces.com
bucket-test2
China (Guangzhou)
tos-s3-cn-guangzhou.volces.com

Obtaining access keys

TOS uses access keys, which are provided by Volcano Engine Identity and Access Management (IAM), for signature authentication. These access keys consist of an access key ID and a secret access key. When accessing TOS with S3-compatible SDKs or tools, set up your credentials using the access key ID and secret access key obtained from IAM. For more information, see Access Keys.

Request methods

TOS only supports virtual-hosted-style requests and does not support path-style requests. Therefore, when accessing TOS through S3-compatible SDKs or tools, make sure that the request method is set to "VirtualHostStyle".

For example, when using the AWS S3 Java SDK, disable path-style access during client initialization by setting: AmazonS3Builder.withPathStyleAccessEnabled(false).

ACL permissions

Although TOS is fully consistent with the S3 protocol in terms of access control list (ACL) permission definitions, it has fewer ACL features than S3. The following table describes the differences.

Feature	AWS S3	TOS
Carrying ACL configuration in the request header (x-amz-acl)	Seven enumeration values are supported: private public-read public-read-write authenticated-read aws-exec-read bucket-owner-read bucket-owner-full-control	Six enumeration values are supported, which are semantically the same as those in S3: private public-read public-read-write authenticated-read bucket-owner-read bucket-owner-full-control
Carrying ACL configuration in the request body	Three enumeration values are supported for the Type field: CanonicalUser AmazonCustomerByEmail Group If Type is Group, three enumeration values are supported for URI: AllUsers: http://acs.amazonaws.com/groups/global/AllUsers AuthenticatedUsers: http://acs.amazonaws.com/groups/global/AuthenticatedUsers LogDelivery: http://acs.amazonaws.com/groups/s3/LogDelivery	Two enumeration values are supported for the Type field, which are semantically the same as those in S3: CanonicalUser Group If Type is Group, two enumeration values are supported for URI, which are semantically the same as those in S3: AllUsers: http://acs.amazonaws.com/groups/global/AllUsers AuthenticatedUsers: http://acs.amazonaws.com/groups/global/AuthenticatedUsers

Storage types

TOS currently supports the following storage types: Standard Storage, Infrequent Access Storage, Intelligent Tiering, Flashback Archive, Archive, Cold Archive, and Deep Cold Archive. All TOS storage types except Deep Cold Archive are supported by S3. The mapping is as follows.

AWS S3	TOS
Nine enumeration values are supported: STANDARD REDUCED_REDUNDANCY STANDARD_IA ONEZONE_IA INTELLIGENT_TIERING GLACIER_IR GLACIER DEEP_ARCHIVE OUTPOSTS	Six enumeration values are supported: STANDARD: Standard Storage IA: Infrequent Access Storage INTELLIGENT_TIERING: Intelligent Tiering ARCHIVE_FR: Flashback Archive ARCHIVE: Archive COLD_ARCHIVE: Cold Archive

warning

When using AWS SDKs, set the object storage type according to the values defined by S3.

When using TOS SDKs, set the object storage type according to the values defined by TOS.

APIs

TOS is fully compatible with common S3 APIs, such as CreateBucket, ListBuckets, PutObject, GetObject, and ListObjects. For more information, see the following table.

warning

Although TOS ensures full compatibility with S3 APIs, TOS APIs typically have fewer features than their S3 counterparts. Some parameters supported by the S3 protocol may be ignored in TOS requests. Please pay special attention to these differences while using TOS.

Appendix: TOS-compatible S3 APIs

API type	S3 API	Compatibility with TOS
Bucket APIs	CreateBucket	Fully compatible.
		HeadBucket	Fully compatible.
		DeleteBucket	Fully compatible.
		ListBuckets	Fully compatible.
Object APIs	CopyObject	This API's request parameters and response structure are compatible with TOS. However, this API can only be used to copy objects within the same region and not across different regions.
		DeleteObject	Fully compatible.
		DeleteObjects	Fully compatible. Its TOS counterpart is named DeleteMultiObjects.
		GetObject	Fully compatible.
		GetObjectAcl	Fully compatible. Its TOS counterpart is named GetObjectACL.
		HeadObject	Fully compatible.
		ListObjects	Fully compatible.
		ListObjectVersions	Fully compatible.
		PutObject	Fully compatible.
		PostObject	Fully compatible.
		PutObjectAcl	Fully compatible. Its TOS counterpart is named PutObjectACL.
Part APIs	CreateMultipartUpload	Fully compatible.
		UploadPart	Fully compatible.
		CompleteMultipartUpload	This API's request parameters and response structure are compatible with TOS. However, this API can only be used to assemble all of the uploaded parts under a specified multipart upload ID, not just some of them.
		AbortMultipartUpload	Fully compatible.
		UploadPartCopy	Fully compatible.
		ListMultipartUploads	Fully compatible.
		ListParts	Fully compatible.

Last updated: 2026.10.13 10:14:57

Bucket	Region	Endpoint
bucket-test1	China (Beijing)	tos-s3-cn-beijing.volces.com
bucket-test2	China (Guangzhou)	tos-s3-cn-guangzhou.volces.com