You can use the cp command to upload local files to TOS.

Command syntax

Linux/macOS

Windows

Upload an object

Upload a folder

Upload a list of objects

Recommendations for optimizing performance

tosutil supports adjusting concurrency via the -j and -p parameters. When you upload large objects in batches, the actual concurrency equals the -j value multiplied by the -p value. Recommendations:

When resources are limited

To prevent performance degradation or failure due to resource exhaustion when resources like network bandwidth or memory are limited, do not set the concurrency level too high. Start with a lower concurrency value and gradually increase it until you find the optimal setting for your resources.

Improving overall QPS and bandwidth

When uploading, downloading, or copying small objects, set a higher concurrency to improve QPS.

When uploading, downloading, or copying large objects, avoid excessively high concurrency to prevent bandwidth contention and performance degradation.

Use the -nfj and -j parameters to manage the concurrency of non-multipart and multipart tasks separately. For example, setting -j=100, -nfj=80 results in an overall concurrency of 100. In this case, the maximum concurrency for non-multipart tasks (typically smaller objects) is 80, and the maximum concurrency for multipart tasks (typically larger objects) is 20.

When using public networks

Uploads may be slow or prone to failure due to unstable public networks and limited client bandwidth. Adjust the following parameters:

Increase the maxRetryCount value in the configuration file to address instability through retries.

If the client's bandwidth is limited, adjust the -j and -p parameters accordingly to prevent exhaustion of the bandwidth and connection failures.

Reduce part size by adjusting the -ps parameter to improve the success rate of individual uploads, downloads, and copies.

Parameters

For descriptions of common optional parameters, see Global options.

Parameter	Type	Description
file_url	String	The path for a local file. tip If you need to upload a list of objects, add multiple file_url values to the command and separate them with commas (,).
folder_url	String	The path for a local folder. This parameter must be used in conjunction with the -r parameter.
bucket	String	Bucket name.
key	String	Object name.
prefix	String	The prefix used for filtering objects in batch operations.
-r	Bool	Perform batch upload, download, or copy operations.
-dryRun	Bool	Run in test mode without executing any actual uploads, downloads, or copy operations.
-link	Bool	Upload the actual target path pointed to by symbolic links for both files and directories, rather than uploading the symbolic link files themselves.
-msm	Enum	Sets the mode for uploading multiple objects or folders at once. The options are as follows: -msm=1: The upload path is a list of objects or folders separated by commas (,). -msm=2: The upload path is a manifest file containing a list of objects or folders. warning Do not use the -msm=1 mode if an object or folder name contains a comma. Unless the -r parameter is set, folders in the list will not be uploaded.
-recover	String	The ID of an object upload, download, or copy task to be resumed in the result file.
-u	Bool	This parameter is used for incremental uploads, downloads, or copies of objects. It compares the source and destination. We recommend using this setting in the following cases: The destination does not exist. The file size at the destination differs from that at the source. The last modified time at the destination is earlier than that at the source.
-vchecksum	Bool	Validate data integrity during uploads or downloads using the CRC-64 algorithm.
-flat	Bool	Do not include relative parent directories when uploading, downloading, or copying objects.
-j	Integer	The maximum concurrency for batch operations. This value is set to 3 by default, which is the defaultJobs value in the configuration file.
-p	Integer	The maximum concurrency for each multipart task. This value is set to 20 by default, which is the defaultParallels value in the configuration file.
-threshold	Integer	The limit, in bytes, for a multipart task. This value is set to the defaultBigfileThreshold value in the configuration file by default. tip This parameter supports configuration with capacity units. For example, there is no need to manually calculate how many bytes are in 1 MB—just write "1 MB," and the program will recognize it.
-nfj	Integer	The maximum concurrency for regular object batch processing (non-multipart tasks). Setting this parameter divides the concurrency specified by the -j parameter into two parts: one for non-multipart tasks (nfj) and one for multipart tasks (j-nfj). This parameter only takes effect during batch operations.
-ps	Integer/String	The size, in bytes, of each part during a multipart task. This value is set to the defaultPartSize value in the configuration file by default. tip This parameter supports configuration with capacity units. For example, there is no need to manually calculate how many bytes are in 1 MB—just write "1 MB," and the program will recognize it.
-cpd	String	The folder where checkpoint files are stored. The default location is specified in the configuration file as defaultCheckpointDir.
-acl	String	Sets the predefined access control lists (ACLs) for objects to be uploaded or copied. Possible values: private public-read public-read-write authenticated-read bucket-owner-read bucket-owner-full-control bucket-owner-entrusted
-sc	Enum	Sets the storage type of objects to be uploaded. The options are as follows: STANDARD: Standard Storage, which is the default value. IA: Infrequent Access Storage. ARCHIVE_FR: Flashback Archive. ARCHIVE: Archive. COLD_ARCHIVE: Cold Archive. DEEP_COLD_ARCHIVE: Deep Cold Archive. tip Cold Archive is only supported in China (Beijing), China (Shanghai), and China (Guangzhou). Deep Cold Archive is only supported in China (Beijing), China (Shanghai), and China (Guangzhou). Deep Cold Archive is currently in closed beta. Please contact your account manager for access.
-meta	String	Sets the user-defined metadata of objects to be uploaded or copied.
-sse	String	Sets the server-side encryption algorithm used when uploading or copying objects. Possible values: aes256 kms This value is set to the defaultSse value in the configuration file by default.
-sseKeyId	String	The key used when uploading or copying objects with server-side encryption with Key Management Service (KMS). This value is set to the defaultSseKeyId value in the configuration file by default.
-contentType	String	Sets the Content-Type header metadata for objects during upload or copy operations.
-contentDisposition	String	Sets the Content-Disposition metadata for objects during upload or copy operations.
-contentEncoding	String	Sets the Content-Encoding metadata for objects during upload or copy operations.
-contentLanguage	String	Sets the Content-Language metadata for objects during upload or copy operations.
-cacheControl	String	Sets the Cache-Control metadata for objects during upload or copy operations.
-expires	String	Sets the Expires metadata for objects during upload or copy operations, in the format of YYYYMMDDHHmmSS.
-include	String	The object matching pattern used to include objects for upload, download, or copying. The following characters have special behavior: ? : Matches any single character within an object name. : Matches zero or more characters within an object name. #: Serves as a delimiter. For example, -include=.png#*.txt matches all objects ending in .png and .txt. tip To avoid parsing failures caused by the operating system escaping special characters, enclose the object matching pattern in quotes. This pattern only takes effect when uploading folders or a list of objects. This pattern applies to the full file path, including the object name and directory structure.
-exclude	String	The object matching pattern used to exclude objects from upload, download, or copy operations. The following characters have special behavior: ? : Matches any single character within an object name. : Matches zero or more characters within an object name. #: Serves as a delimiter. For example, -exclude=.png#*.txt matches all objects ending in .png and .txt. tip To avoid parsing failures caused by the operating system escaping special characters, enclose the object matching pattern in quotes. This pattern only takes effect when uploading folders or a list of objects. This pattern applies to the full file path, including the object name and directory structure.
-timeRange	String	Sets the time range for the last modified time of objects during upload, download, or copy, in the format of YYYYMMDDHHmmSS-YYYYMMDDHHmmSS.
-at	Bool	Upload a list of objects whose last access time falls within the -timeRange.
-ddo	Bool	Do not upload the folder itself as an object.
-dmf	Bool	Do not apply the parameters, -include/-exclude and -timeRange, to folders.
-arcDir	String	The archive folder for successfully uploaded objects.
-fr	Bool	Generate a result file when working with a single object.
-o	String	The folder where result files are stored. The default location is specified in the configuration file as defaultOutputDir.
-bt	String	The type of the bucket. The options are as follows: fns: A flat namespace (FNS) bucket. hns: A hierarchical namespace (HNS) bucket. If the bucket type is not specified, the default will be the type obtained from the bucket metadata.

Examples

Upload an object

Use the following command to upload the object object from the /Users/Downloads/TOS/tosutil folder to the bucketname bucket. The uploaded object will be named object.

Command

./tosutil cp /Users/Downloads/TOS/tosutil/object tos://bucketname/object

Output

Start at 2022-10-27 10:49:12.407874 +0000 UTC

Parallel: 1 Jobs: 5
Threshold: 100.00MB PartSize: auto
VerifyChecksum: false
CheckpointDir: /.tosutil_checkpoint

[-------------------------------------------] 100.00% ?/s 50.50KB/50.50KB 162ms

Upload successfully, 50.50KB, n/a, /Users/Downloads/TOS/tosutil/object --> tos://bucketname/object, cost [164], status [200], request id [2643485a6228acbb635a6228-ac1173a1-1oo0Rk-PuO-cb-tos-bj-3]

Upload a folder

Use the following command to upload all objects in the /Users/Downloads/TOS/tosutil folder to the bucketname bucket. The uploaded folder will be named cpfile.

Command

./tosutil cp /Users/Downloads/TOS/tosutil tos://bucketname/cpfile -r

Output

Start at 2022-10-27 10:50:43.872787 +0000 UTC

Parallel: 1 Jobs: 5
Threshold: 100.00MB PartSize: auto
VerifyChecksum: false
CheckpointDir: /Users/.tosutil_checkpoint
OutputDir: /Users/.tosutil_output

[----------------------------] 100.00% tps:0.00 ?/s 4/4 181.45KB/181.45KB 183ms
Succeed count is: 4 Failed count is: 0
Succeed bytes is: 181.45KB
Metrics [max cost:178 ms, min cost:103 ms, average cost:143.75 ms, average tps:21.62]

Task id is: 34c8480e-55e5-11ed-806b-aa665a3c7f26

Torch Object Storage