Docs

Torch Object Storage

Best Practices

Overview of best practices

This topic provides an overview of the best practices for using TOS to more effectively address business demands.

Accessing TOS with third-party tools

Topic	Description
Accessing TOS with Rclone	Rclone is a powerful open-source command-line tool that seamlessly integrates with various cloud storage systems, including S3-compatible object storage. With Rclone, you can easily upload, download, synchronize, and manage your data.
Managing TOS resources with S3 Browser	S3 Browser is a free Windows client for Amazon S3. TOS is compatible with the S3 protocol, so you can manage your TOS resources using S3 Browser.
Accessing TOS with Flutter SDK	The TOS Flutter SDK is now available on pub.dev. Use it to access TOS and upload or download your data.

Granting temporary access

Topic	Description
Using pre-signed URLs for secure data downloads	Applications such as video surveillance, mobile apps, and web platforms often require uploading or downloading files, such as images, documents, and videos, to or from the server. To address such scenarios, TOS offers a secure solution for data downloads using pre-signed URLs. This topic outlines how to generate these URLs and describes how clients can send requests to access them.
Accessing TOS with temporary credentials of Security Token Service	For security purposes, we recommend using temporary credentials instead of permanent ones in the client code when files are uploaded to TOS directly from mobile applications or PC clients. Specifically, the application server obtains the access key ID (AK) and secret access key (SK) via the STS mechanism and delivers the temporary credentials to the client.

Data migration

Topic	Description
Online migration with zero service disruption	If your business relies on third-party services and needs to transition to TOS, maintaining service stability and preventing any negative operational impact during the data migration process is essential.

Mounting buckets

Topic	Description
Mounting TOS buckets on Windows	To use TOS as a local drive, you can mount TOS buckets directly onto a Windows operating system.
Mounting TOS buckets on Linux	To use TOS as a drive, you can mount TOS buckets directly onto a Linux operating system.

Data processing

Topic	Description
Combining redirection rules and image processing	During the business data migration, image distribution switched to TOS. However, the incremental images have not yet been migrated to TOS. You can use the TOS redirection feature to retrieve these images from the origin server. After storing these images in TOS, you can apply image processing parameters to them.

Checking data integrity

Topic	Description
Verifying the integrity of uploaded objects	Cybersecurity incidents such as network hijacking or data caching outside the TOS environment may cause data inconsistencies when you upload data to TOS from the Internet. TOS offers multiple features to ensure data integrity. You can use Content-MD5, Content-SHA256, or CRC64 to guarantee data integrity during uploads.
Verifying the integrity of downloaded objects	Cybersecurity incidents such as network hijacking or data caching outside the TOS environment may cause data inconsistencies when you download data from TOS over the Internet. You can use CRC64 to check data integrity during downloads.

Performance optimization

Topic	Description
Best practices for optimizing performance	If you use sequential prefixes in your naming convention for objects (e.g., timestamps or alphabetical order), requests may concentrate on a specific shard. This creates hotspots. These hotspots are subject to the storage medium's read/write upper limits, which may increase access delays. You can apply the methods discussed in this topic to improve program performance.
Benchmarking TOS performance using COSBench	COSBench (Cloud Object Storage Benchmark) is a benchmarking tool developed by the Intel team using Java to measure the performance of object storage services. You can use COSBench to conduct stress tests on the read/write performance of TOS. This will enable you to benchmark its performance and ensure business stability.

Others

Topic	Description
Compatibility with the Amazon S3 protocol	TOS is compatible with the Amazon S3 protocol. Once data is migrated from any S3-compatible storage solution to TOS, applications that previously accessed that storage solution can seamlessly switch to TOS with minimal adjustments.
Uploading data from WeChat Mini Programs to TOS	WeChat Mini Programs, which can be used without downloading, are attracting a growing number of developers and users. One common concern for developers is how to upload files from a WeChat Mini Program to TOS.
Uploading data from web browsers to TOS	You can use the PostObject API to upload content directly from a web browser as an HTML form to TOS. The signature required for the upload can be obtained from the server.
Accessing TOS using PrivateLink	PrivateLink allows you to access TOS via a dedicated line from an on-premises Internet data center (IDC).
Accessing and storing TOS data in PyTorch jobs	You can access and store TOS data in PyTorch jobs.
Creating VeArk batch inference tasks with TOS	Volcano Engine Ark (VeArk) is an enterprise-grade, developer-centric platform for large language models (LLMs). It offers a wide range of features for training, inferencing, evaluating, and fine-tuning models, playing a pivotal role in the LLM ecosystem. You can create VeArk batch inference tasks with TOS.
Tracking download behaviors with TOS Browser	TOS Browser provides a link tracking feature that allows you to track download behaviors after objects are shared.
Accessing TOS through an ECS reverse proxy	TOS is usually accessed via domain names, and its service addresses change dynamically. For this reason, fixed IP addresses are not provided. However, certain industries may have security and compliance requirements that necessitate accessing TOS through fixed IP addresses. To meet these needs, you can set up an Nginx reverse proxy server on an Elastic Compute Service (ECS) instance. Then, you can access the objects in your TOS bucket via the public IP address of port 80 on the ECS instance. This gives you access to your TOS resources via a fixed IP address.

Last updated: 2026.09.16 10:50:51