Torch Object Storage
Torch Object Storage
Torch Object Storage
Document
What's New
Announcements
Billing
Billable items
Billing method
Getting Started
User Guide
Working with buckets
Basic configurations
Managing lifecycle
Back-to-origin rules
Event notification
Intelligent Tiering
Permissions
Domain management
Fault tolerance and disaster recovery
Cross-region replication
Same-region replication
Data redundancy
Image processing
ObjectSet
Working with objects
Managing folders
Monitoring and alerting
Metric monitoring
Access control
Bucket policies
Common configuration scenarios
Authorizing IAM users of the current account
Authorizing other accounts
Managing access via tools and SDKs
Managing public read access
Data security
Access points
Multi-region access points (MRAP)
Single-region access points (SRAP)
Converged access points (CAP)
DevMind
TOS accelerator
SenseFlow
TOS Agent
Data processing
Basic concepts
Image processing
Media processing
Document preview
Task processing
Data subscription
Flow control management
Quality of Service (QoS) policies
Hierarchical namespace (HNS)
User guide
Tool reference
Best practices
TOS Vectors
User guide
API reference
Vector bucket
Vector index
SDK reference
Python-Vectors
Vector bucket
Vector index
Java-Vectors
Vector bucket
Vector index
Node.js-Vectors
Vector bucket
Vector index
Tool Guide
tosutil
Common commands
Upload, download, and copy objects
Share objects
Auxiliary commands
Common examples
TOS Browser
FSX
FSX best practices
TOS PyTorch Connector
tosfsspec
API Reference
Signature Mechanism
The header contains a signature
Bucket
Basic Bucket Operations
Bucket Lifecycle
Access Monitor
Bucket ACL
Bucket CORS
Bucket Policy
Back-to-origin Rules
Bucket Replication
Versioning
Static Website
Event Notification
Custom Domain
Server-side encryption
Transfer Acceleration
Data Processing
Image processing
Media processing
Video transcoding
Audio transcoding
video to GIF
Document preview
File decompression
Log Storage
Retention Policy
Requester pays
TLS version
Object
Basic Object Operations
Object ACL
Fetch Object
Symbolic Link
WORM object
Multipart Upload
SDK Reference
Java
Pre-signing Mechanism
Bucket Interface
Basic Bucket Interface
Manage Bucket Configuration
Object Interface
Upload Object
Download Object
Upload and Download Speed Limit
Copy Object
List Object
Data Processing
Image Processing
Video Processing
Audio Processing
Log Configuration
Hierarchical Namespace
Bucket base interface
Introduction to Typical Scenes
Python
Data processing
Pre-signature mechanism
Bucket interface
Basic bucket interface
Manage bucket configuration
Object interface
Upload object
Download object
Copy object
List objects
Introduction to typical scenes
C
Object Interface
Upload object
C++
Pre-signature Mechanism
Bucket Interface
Basic Bucket Interface
Manage Bucket Configuration
Object Interface
Upload Object
Download Object
Copy Object
List Objects
Data Processing
Go
Object Interface
Upload Object
Download Object
Copy Object
List Objects
Manage Objects
Bucket Interface
Manage Bucket Configuration
Data Processing
Log Configuration
Introduction to Typical Scenes
Pre-signature Mechanism
Node.js
Pre-signature Mechanism
Bucket Interface
Basic Bucket Interface
Manage Bucket Configuration
Object Interface
Upload object
Download object
Copy object
List objects
Data Processing
Image Processing
Introduction to Typical Scenes
Browser.js
Pre-signature Mechanism
Upload Object
Download Object
Copy Object
List Objects
Manage Objects
Data Processing
.NET
Bucket Interface
Object Interface
Upload Object
Download Object
List Objects
Introduction to Typical Scenes
PHP
Bucket Interface
Object Interface
Download Object
Data Processing
Exception Handling
Android
Bucket Interface
Object Interface
Upload Object
Download Object
Exception Handling
iOS
Bucket Interface
Object Interface
Upload Object
Download Object
Harmony
Pre-signature Mechanism
Bucket Interface
Object Interface
Upload Object
Download Object
List Objects v2
Introduction to Typical Scenarios
Rust
Pre-signing mechanism
Bucket interface
Basic bucket operations
Lifecycle rules
Bucket ACLs
Cross-origin resource sharing
Bucket inventory
Back-to-origin rules
Bucket replication
Static website hosting
Event notification
Custom domain
Server-side encryption
Object interface
Basic operation
Access control
Error Codes
01-SERVICE_ERROR
02-AUTH
03-ACCESS_CONTROL
04-QOS
05-REQUEST
06-BUCKET
07-LIFECYCLE
08-ACL
09-CORS
10-POLICY
11-MIRROR
12-REPLICATION
13-WEBSITE
14-NOTIFICATION
15-ENCRYPT
16-CUSTOMDOMAIN
17-OBJECT
18-TAG
19-MULTIPART
20-STORAGE_CLASS
21-CALLBACK
22-DATA_PROCESS
23-BATCH
24-STORAGE_LENS
25-INVENTORY
26-DIRECTORY_BUCKET
FAQ
Best Practices
AIGC-Doubao LLMs
Accessing TOS with third-party tools
Granting temporary access
Data migration
Data processing
Checking data integrity
Performance optimization
- Documentation
Torch Object StorageSDK ReferenceC++Object InterfaceDownload ObjectResumable Download (C++ SDK)
Copy page
Download PDFDownload Object
Resumable Download (C++ SDK)
Copy pageDownload PDFResumable Download (C++ SDK)
When downloading large objects, network fluctuations and similar issues may result in download failures. The TOS C++ SDK provides a retry mechanism to enhance the stability of object downloads; however, it is possible that, even after multiple retries, the download may still not be completed. To address the aforementioned situation, the TOS C++ SDK provides a resumable download feature, utilizing a local CheckPoint mechanism to record successfully downloaded file segments. If a download is interrupted due to network anomalies or machine failures, this interface can be invoked again to resume the download.
Resumable download divides the target object into multiple segments and supports concurrent downloading. Once all segments have been downloaded, they are merged into a complete file. You can configure the chunk size for resumable downloads, the number of threads for downloading chunks, client-side download speed limits, event callback functions, etc. It is also possible to cancel the task during the execution of a resumable download task.
Precautions
- Before downloading an object, you must have the tos:GetObject permission. For detailed instructions, please refer to the Permission configuration guide.
- For buckets with versioning enabled, you must have the tos:GetObjectVersion permission to download a specified version of an object. For detailed instructions, please refer to the Permission configuration guide.
- If an application downloads the same object in large quantities simultaneously, your access speed will be restricted by TOS bandwidth and regional limitations. It is recommended that you utilize CDN products to enhance performance while simultaneously reducing your costs. For detailed information on accessing TOS via CDN, please refer to Using CDN to accelerate access to TOS Resources.
- The SDK records the download status information in the Checkpoint file, so the program must have write permissions for the Checkpoint file.
- When performing a resumable download, the progress information of the file is recorded in the Checkpoint file. If a segment fails to download during the process, the subsequent download will resume from the point recorded in the Checkpoint file, thereby achieving the effect of download resumption. After the download is complete, the Checkpoint file will be deleted.
- If, during the download process, the file's ETag changes or any part is missing or modified, the file will be re-downloaded.
Sample code
Resumable download
The following code is used to perform a resumable download of the object exampledir/exampleobject.txt from the bucket examplebucket to a local file, and to retry the download in case of failure. If a network timeout error occurs during the download process, you can call downloadFile with the same parameters to resume the download from the breakpoint and re-enter the process.
#include "TosClientV2.h"
using namespace VolcengineTos;
int main(void){
std::string region = "Your Region";
std::string accessKey = std::getenv("TOS_ACCESS_KEY");
std::string secretKey = std::getenv("TOS_SECRET_KEY");
std::string bucketName = "examplebucket";
std::string objectName = "exampledir/exampleobject.txt";
std::string filePath = "/localpath/examplefile.txt";
std::string CheckpointFilePath = "yourCheckpointFilepath";
InitializeClient();
TosClientV2 client(region, accessKey, secretKey);
DownloadFileInput input(bucketName, objectName);
input.setTaskNum(10);
input.setEnableCheckpoint(true);
input.setPartSize(20 * 1024 * 1024);
input.setFilePath(filePath);
auto output = client.downloadFile(input);
if (!output.isSuccess()) {
std::cout << "DownloadFile failed." << output.error().String() << std::endl;
CloseClient();
return -1;
}
std::cout << "DownloadFile success. object etag is: " << output.result().getHeadObjectV2Output().getETags() << std::endl;
CloseClient();
return 0;
}
Configure progress bar
The following code configures the resumable download progress bar functionality.
#include "TosClientV2.h"
using namespace VolcengineTos;
static void ProgressCallback(std::shared_ptr<DataTransferStatus> datatransferstatus) {
int64_t consumedBytes = datatransferstatus->consumedBytes_;
int64_t totalBytes = datatransferstatus->totalBytes_;
int64_t rwOnceBytes = datatransferstatus->rwOnceBytes_;
DataTransferType type = datatransferstatus->type_;
int64_t rate = 100 * consumedBytes / totalBytes;
std::cout << "rate:" << rate << ","
<< "ConsumedBytes:" << consumedBytes << ","
<< "totalBytes:" << totalBytes << ","
<< "rwOnceBytes:" << rwOnceBytes << ","
<< "DataTransferType:" << type << std::endl;
}
int main(void){
std::string region = "Your Region";
std::string accessKey = std::getenv("TOS_ACCESS_KEY");
std::string secretKey = std::getenv("TOS_SECRET_KEY");
std::string bucketName = "examplebucket";
std::string objectName = "exampledir/exampleobject.txt";
std::string filePath = "/localpath/examplefile.txt";
std::string CheckpointFilePath = "yourCheckpointFilepath";
InitializeClient();
TosClientV2 client(region, accessKey, secretKey);
DownloadFileInput input(bucketName, objectName);
input.setTaskNum(10);
input.setEnableCheckpoint(true);
input.setPartSize(20 * 1024 * 1024);
input.setFilePath(filePath);
DataTransferListener processHandler = {ProgressCallback, nullptr};
input.setDataTransferListener(processHandler);
auto output = client.downloadFile(input);
if (!output.isSuccess()) {
std::cout << "DownloadFile failed." << output.error().String() << std::endl;
CloseClient();
return -1;
}
std::cout << "DownloadFile success. object etag is: " << output.result().getHeadObjectV2Output().getETags() << std::endl;
CloseClient();
return 0;
}
Handle event callbacks
During the invocation of the downloadFile API, events such as temporary file creation, file segment download, and temporary file renaming are triggered. You may pass in a custom interface to monitor these download-related events and implement your own business logic.
The following code is used to customize the callback function for resumable downloads.
#include "TosClientV2.h"
using namespace VolcengineTos;
static void DownloadCallBack(std::shared_ptr<DownloadEvent> event) {
DownloadEventType type_ = event->type_;
bool error_ = event->error_;
std::string bucket_ = event->bucket_;
std::string key_ = event->key_;
std::string versionId_ = event->versionId_;
std::string filePath_ = event->filePath_;
std::string fileInfo = bucket_ + key_ + versionId_ + filePath_;
std::shared_ptr<std::string> checkpointFile_ = event->checkpointFile_;
std::shared_ptr<std::string> tempFilePath_ = event->tempFilePath_;
std::shared_ptr<DownloadPartInfo> downloadPartInfo_ = event->downloadPartInfo_;
std::string downloadPartInfo = "";
if (downloadPartInfo_ != nullptr) {
downloadPartInfo = std::to_string(downloadPartInfo_->rangeStart_) + "," +
std::to_string(downloadPartInfo_->rangeEnd_) + "," +
std::to_string(downloadPartInfo_->partNumber_);
}
std::cout << "type:" << type_ << ","
<< "error:" << error_ << ","
<< "fileInfo:" << fileInfo << ","
<< "checkpointFile:" << *checkpointFile_ << ","
<< "tempFilePath:" << *tempFilePath_ << ","
<< "downloadPartInfo:" << downloadPartInfo << std::endl;
}
int main(void){
std::string region = "Your Region";
std::string accessKey = std::getenv("TOS_ACCESS_KEY");
std::string secretKey = std::getenv("TOS_SECRET_KEY");
std::string bucketName = "examplebucket";
std::string objectName = "exampledir/exampleobject.txt";
std::string filePath = "/localpath/examplefile.txt";
std::string CheckpointFilePath = "yourCheckpointFilepath";
InitializeClient();
TosClientV2 client(region, accessKey, secretKey);
DownloadFileInput input(bucketName, objectName);
input.setTaskNum(10);
input.setEnableCheckpoint(true);
input.setPartSize(20 * 1024 * 1024);
input.setFilePath(filePath);
DownloadEventListener downloadHandler = {DownloadCallBack};
input.setDownloadEventListener(downloadHandler);
auto output = client.downloadFile(input);
if (!output.isSuccess()) {
std::cout << "DownloadFile failed." << output.error().String() << std::endl;
CloseClient();
return -1;
}
std::cout << "DownloadFile success. object etag is: " << output.result().getHeadObjectV2Output().getETags() << std::endl;
CloseClient();
return 0;
}
Manage client-side rate limiting
The following code is used to set the speed limit for a resumable download client.
#include "TosClientV2.h"
using namespace VolcengineTos;
int main(void){
std::string region = "Your Region";
std::string accessKey = std::getenv("TOS_ACCESS_KEY");
std::string secretKey = std::getenv("TOS_SECRET_KEY");
std::string bucketName = "examplebucket";
std::string objectName = "exampledir/exampleobject.txt";
std::string filePath = "/localpath/examplefile.txt";
std::string CheckpointFilePath = "yourCheckpointFilepath";
InitializeClient();
TosClientV2 client(region, accessKey, secretKey);
DownloadFileInput input(bucketName, objectName);
input.setTaskNum(10);
input.setEnableCheckpoint(true);
input.setPartSize(20 * 1024 * 1024);
input.setFilePath(filePath);
std::shared_ptr<RateLimiter> RateLimiter(NewRateLimiter(20 * 1024 * 1024, 5 * 1024 * 1024));
input.setRateLimiter(RateLimiter);
auto output = client.downloadFile(input);
if (!output.isSuccess()) {
std::cout << "DownloadFile failed." << output.error().String() << std::endl;
CloseClient();
return -1;
}
std::cout << "DownloadFile success. object etag is: " << output.result().getHeadObjectV2Output().getETags() << std::endl;
CloseClient();
return 0;
}
Pause or cancel resumable download
The downloadFile interface allows you to proactively pause or cancel a resumable transfer task. Pausing a resumable download task will stop the download. When downloadFile is invoked again, the download can resume from the position where it was previously paused.
The following code demonstrates how to pause or cancel a resumable transfer.
#include "TosClientV2.h"
#include <thread>
using namespace VolcengineTos;
static void threadFunction1(TosClientV2 client, DownloadFileInput input) {
auto output = client.downloadFile(input);
if (!output.isSuccess()) {
std::cout << "DownloadFile failed." << output.error().String() << std::endl;
}
std::cout << "DownloadFile success. object etag is: " << output.result().getHeadObjectV2Output().getETags() << std::endl;
}
static void threadFunction2(std::shared_ptr<CancelHook> CancelHook) {
CancelHook->Cancel(false);
}
int main(void){
std::string region = "Your Region";
std::string accessKey = std::getenv("TOS_ACCESS_KEY");
std::string secretKey = std::getenv("TOS_SECRET_KEY");
std::string bucketName = "examplebucket";
std::string objectName = "exampledir/exampleobject.txt";
std::string filePath = "/localpath/examplefile.txt";
std::string CheckpointFilePath = "yourCheckpointFilepath";
InitializeClient();
TosClientV2 client(region, accessKey, secretKey);
DownloadFileInput input(bucketName, objectName);
input.setTaskNum(10);
input.setEnableCheckpoint(true);
input.setPartSize(20 * 1024 * 1024);
input.setFilePath(filePath);
std::shared_ptr<CancelHook> CancelHook(NewCancelHook());
input.setCancelHook(CancelHook);
std::thread thread1(threadFunction1, client, input);
thread1.detach();
TimeUtils::sleepSecondTimes(5);
std::thread thread2(threadFunction2, CancelHook);
thread1.join();
thread2.join();
CloseClient();
return 0;
}
Last updated: 2026.07.27 10:23:49
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
