Docs

Torch Object Storage

tosfsspec

Troubleshooting

This topic introduces the most common issues that you might encounter when using tosfsspec, along with the methods for troubleshooting them.

Troubleshooting methods

If you encounter a problem, you can enable debug logging for fsspec and tosfsspec to help identify the cause of the issue. The following example can be used directly in your code:

This code provides clear details about operations and network requests, which will help you analyze the problem.

FAQ

ValueError: The part_size must be >= 4194304

Symptoms:

A ValueError occurs during the initialization of TosFileSystem or when the fs.open() method is called, indicating that part_size or block_size must be greater than or equal to 4 MB.

What it means:

TOS has a minimum size limit for parts of multipart uploads. In write mode ('wb' or 'ab'), the block_size parameter determines the size of each part. If the size is less than 4 MB, a ValueError is returned.

How to fix it:

When using fs.open(..., mode='wb') or fs.open(..., mode='ab'), set block_size to a value of at least 4 * 1024 * 1024.

When initializing TosFileSystem(part_size=...), set part_size to a value of at least 4 * 1024 * 1024.

If you are unsure of the appropriate value, do not set the block_size parameter explicitly. Allow tosfsspec to use the default value of 8 MB.

Permission errors (e.g., TosException: status=403)

Symptoms:

You received a tos.exceptions.TosException with a status code of 403 (Forbidden) when performing operations such as ls, open, or put.

What it means:

Invalid AK/SK: The key (access key ID) or secret (secret access key) provided is incorrect or has expired.

Insufficient IAM permissions: The Identity and Access Management (IAM) policies associated with your account do not authorize this operation. For example, your IAM policies allow read access (tos:GetObject), but do not permit write (tos:PutObject) or list (tos:ListObjects) actions.

IP allowlisting: The bucket or account has an IP allowlist configured, and your source IP address is not within the allowed range.

How to fix it:

Verify AK/SK: Make sure that there are no leading or trailing spaces, that the keys are enabled, and that they match the account being used.

Check IAM policies: Make sure the required permissions are included. Common permissions include read (tos:GetObject), write (tos:PutObject), and list (tos:ListObjects).

Verify your egress IP: Check if it is allowed by the bucket's access control policies.

Directory operations not working as expected

Symptoms:

Objects exist under the specified path. However, fs.isdir("tos://my-bucket/some/path/") returns False, indicating that the path is not a directory.

The directory is empty, yet you cannot delete it using fs.rmdir().

What it means:

In TOS, flat namespace (FNS) buckets lack a true directory hierarchy. Instead, directories are simulated using zero-byte objects whose names end with a trailing slash (/).

isdir checks whether an object named some/path/ exists. If the object does not exist, even if some/path/file.txt exists, isdir will return False.

rmdir can only delete a zero-byte object that represents a directory. Additionally, no other objects can exist under the directory, meaning no objects can have that directory as their prefix.

How to fix it:

Use fs.makedirs(path, exist_ok=True) to explicitly create the directory, ensuring that the directory marker object is created.

A simpler way to check if a path points to a directory is to see if its name ends with /, or to use fs.info(path)['type'] == 'directory'.

To delete a non-empty directory, use fs.rm(path, recursive=True).

ValueError: The maxdepth is not supported for hns bucket

Symptoms:

A ValueError occurs when the maxdepth parameter is passed to a bucket during recursive operations, such as walk(), glob(), or rm(recursive=True).

What it means:

The maxdepth parameter controls the depth of recursive traversal and only applies to FNS buckets. Hierarchical namespace (HNS) buckets have a native directory hierarchy. Therefore, the underlying traversal APIs do not support depth limits.

How to fix it:

Confirm your bucket type.

If it is an HNS Bucket, remove the maxdepth parameter. Handle the depth limit within your business logic yourself, such as by adding a counter to the walk loop.

If you need to use the maxdepth parameter, consider migrating your data to an FNS bucket.

FileNotFoundError

Symptoms:

A FileNotFoundError occurs when you try to access a path that should exist.

What it means:

Non-existent path: The bucket name or object key is misspelled.

Insufficient permissions: You may receive a "not found" error if you lack the tos:GetObject or tos:ListObjects permissions for the object.

Missing directory marker: fs.info() may return a FileNotFoundError if the specified path is a directory, but the corresponding directory marker object (a zero-byte object that ends with a /) does not exist.

How to fix it:

Use fs.ls() to list the parent paths one level at a time and confirm that each level of the path (bucket, prefix, or object name) is correct.

Check and add the required IAM permissions.

For directory operations, use fs.ls(path) to check the contents instead of relying solely on fs.info(path).

Last updated: 2026.03.31 18:14:37