Docs

Torch Object Storage

Overview

Usage notes

Before using the hierarchical namespace feature, please read this document carefully to ensure proper functionality.

Prerequisites

For a list of regions that support hierarchical namespace, see Supported regions.

Using hierarchical metadata

HNS buckets use a new hierarchical architecture to organize metadata that features directory entities. While supporting the FNS bucket APIs, the system has optimized certain access behaviors for HNS buckets. However, accessing TOS in an unrecommended way may affect performance and user experience. To avoid this, we recommend paying attention to the following points and using the recommended methods to access HNS buckets.

Recommended practices

Using a hierarchical listing with the directory separator "/" as the only delimiter provides excellent performance.

To hide files from others during the writing process, create a temporary file, append content, and then rename the file to its final name.

If you need to speed up the upload of a single file by uploading it concurrently, use the multipart upload feature.

Unrecommended practices

Using a flat listing, (i.e., a listing without a delimiter) is not recommended.

Overwriting a directory is not allowed. Although an attempt to create a directory that already exists may appear successful, the directory will not actually be created.

TOS ensures eventual data consistency for HNS buckets. However, renaming parent directories during object upload makes the original file path (object key) inaccessible after upload. This requires using the new parent directory to reconstruct the object key. The same applies if the parent directory is renamed before the multipart upload is complete.

Using third-party, S3-compatible tools with HNS buckets is not recommended.

Bucket type	Management method	Entity type	Impact
HNS bucket	Manages metadata "hierarchically".	Distinguishes between directory and file entities.	Simulating a directory hierarchy with object keys is different from an actual directory hierarchy system. Behaviors such as deleting directories and listing subdirectories and files also differ between the two bucket types.
FNS bucket	Manages metadata in a "flat" manner.	Treats objects as the only entity.

For example:

The Hadoop S3A component sends directory deletion requests, which can cause server conflicts and integration failures.

Tools that mount buckets as local file systems via the S3 protocol can mistakenly treat directories as files, resulting in errors. If you are using third-party, S3-compatible tools, consult technical support first.

Constraints and limitations

Category	Description
Quantity	You can create up to 100 HNS and FNS buckets per account.
Bucket name	For bucket naming constraints, see Bucket.
Directory name	A forward slash (/) is the fixed delimiter used between directories, subdirectories, and files. Only this can be used as the delimiter when listing.
Object name	The maximum length of a full path is 1,024 characters, and it can have up to 512 directory levels.
Per-bucket limits	Each bucket has a QPS limit of 300,000. If you need a higher QPS, plan ahead by splitting your workloads across multiple buckets to balance the load. Each bucket can contain up to 500 million directories. To exceed this limit, contact technical support to request an extension. There is no limit to the number of objects that can be stored in a single bucket or directory. However, storing too many objects in a single directory is not recommended, as it may affect the directory's concurrent throughput.
EMR Proton version	HNS buckets are supported in EMR Proton 2.0 and later versions. If your Proton version is earlier than 2.0, submit a ticket to contact Volcano Engine Technical Support to request an update to a later version of Proton. For Proton version release notes, see Proton versions.

Last updated: 2026.09.04 11:02:54