Torch Object Storage
Torch Object Storage
Torch Object Storage
Document
What's New
Announcements
Billing
Billable items
Billing method
Getting Started
User Guide
Working with buckets
Basic configurations
Managing lifecycle
Back-to-origin rules
Event notification
Intelligent Tiering
Permissions
Domain management
Fault tolerance and disaster recovery
Cross-region replication
Same-region replication
Data redundancy
Image processing
ObjectSet
Working with objects
Managing folders
Monitoring and alerting
Metric monitoring
Access control
Bucket policies
Common configuration scenarios
Authorizing IAM users of the current account
Authorizing other accounts
Managing access via tools and SDKs
Managing public read access
Data security
Access points
Multi-region access points (MRAP)
Single-region access points (SRAP)
Converged access points (CAP)
DevMind
TOS accelerator
SenseFlow
TOS Agent
Data processing
Basic concepts
Image processing
Media processing
Document preview
Task processing
Data subscription
Flow control management
Quality of Service (QoS) policies
Hierarchical namespace (HNS)
User guide
Tool reference
Best practices
TOS Vectors
User guide
API reference
Vector bucket
Vector index
SDK reference
Python-Vectors
Vector bucket
Vector index
Java-Vectors
Vector bucket
Vector index
Node.js-Vectors
Vector bucket
Vector index
Tool Guide
tosutil
Common commands
Upload, download, and copy objects
Share objects
Auxiliary commands
Common examples
TOS Browser
FSX
FSX best practices
TOS PyTorch Connector
tosfsspec
API Reference
Signature Mechanism
The header contains a signature
Bucket
Basic Bucket Operations
Bucket Lifecycle
Access Monitor
Bucket ACL
Bucket CORS
Bucket Policy
Back-to-origin Rules
Bucket Replication
Versioning
Static Website
Event Notification
Custom Domain
Server-side encryption
Transfer Acceleration
Data Processing
Image processing
Media processing
Video transcoding
Audio transcoding
video to GIF
Document preview
File decompression
Log Storage
Retention Policy
Requester pays
TLS version
Object
Basic Object Operations
Object ACL
Fetch Object
Symbolic Link
WORM object
Multipart Upload
SDK Reference
Java
Pre-signing Mechanism
Bucket Interface
Basic Bucket Interface
Manage Bucket Configuration
Object Interface
Upload Object
Download Object
Upload and Download Speed Limit
Copy Object
List Object
Data Processing
Image Processing
Video Processing
Audio Processing
Log Configuration
Hierarchical Namespace
Bucket base interface
Introduction to Typical Scenes
Python
Data processing
Pre-signature mechanism
Bucket interface
Basic bucket interface
Manage bucket configuration
Object interface
Upload object
Download object
Copy object
List objects
Introduction to typical scenes
C
Object Interface
Upload object
C++
Pre-signature Mechanism
Bucket Interface
Basic Bucket Interface
Manage Bucket Configuration
Object Interface
Upload Object
Download Object
Copy Object
List Objects
Data Processing
Go
Object Interface
Upload Object
Download Object
Copy Object
List Objects
Manage Objects
Bucket Interface
Manage Bucket Configuration
Data Processing
Log Configuration
Introduction to Typical Scenes
Pre-signature Mechanism
Node.js
Pre-signature Mechanism
Bucket Interface
Basic Bucket Interface
Manage Bucket Configuration
Object Interface
Upload object
Download object
Copy object
List objects
Data Processing
Image Processing
Introduction to Typical Scenes
Browser.js
Pre-signature Mechanism
Upload Object
Download Object
Copy Object
List Objects
Manage Objects
Data Processing
.NET
Bucket Interface
Object Interface
Upload Object
Download Object
List Objects
Introduction to Typical Scenes
PHP
Bucket Interface
Object Interface
Download Object
Data Processing
Exception Handling
Android
Bucket Interface
Object Interface
Upload Object
Download Object
Exception Handling
iOS
Bucket Interface
Object Interface
Upload Object
Download Object
Harmony
Pre-signature Mechanism
Bucket Interface
Object Interface
Upload Object
Download Object
List Objects v2
Introduction to Typical Scenarios
Rust
Pre-signing mechanism
Bucket interface
Basic bucket operations
Lifecycle rules
Bucket ACLs
Cross-origin resource sharing
Bucket inventory
Back-to-origin rules
Bucket replication
Static website hosting
Event notification
Custom domain
Server-side encryption
Object interface
Basic operation
Access control
Error Codes
01-SERVICE_ERROR
02-AUTH
03-ACCESS_CONTROL
04-QOS
05-REQUEST
06-BUCKET
07-LIFECYCLE
08-ACL
09-CORS
10-POLICY
11-MIRROR
12-REPLICATION
13-WEBSITE
14-NOTIFICATION
15-ENCRYPT
16-CUSTOMDOMAIN
17-OBJECT
18-TAG
19-MULTIPART
20-STORAGE_CLASS
21-CALLBACK
22-DATA_PROCESS
23-BATCH
24-STORAGE_LENS
25-INVENTORY
26-DIRECTORY_BUCKET
FAQ
Best Practices
AIGC-Doubao LLMs
Accessing TOS with third-party tools
Granting temporary access
Data migration
Data processing
Checking data integrity
Performance optimization
- Documentation
Torch Object StorageSDK ReferencePythonInitialize the client (Python SDK)
Copy page
Download PDFPython
Initialize the client (Python SDK)
Copy pageDownload PDFInitialize the client (Python SDK)
TosClient offers a series of interfaces to interact with TOS services, managing resources such as buckets and objects on TOS services. TosClient can be initialized using AccessKey/SecretKey, STS (Security Token Service), and other methods. During initialization, optional parameters such as connection timeout, retry count on timeout, and maximum idle connections can be set. This article explains how to initialize the TOS Python SDK.
Prerequisites
Notes
In single-process, multi-threaded scenarios, it is recommended to use a single TosClient to avoid repeated initialization of TosClient. In multi-process scenarios, multiple TosClients need to be created and cannot share the same TosClient, otherwise there will be multi-process concurrency security issues, which may lead to data corruption in read and write requests.
Configure long-term access credentials
Before initializing the SDK, configure long-term access credentials first. This section explains the steps to configure long-term access credentials on different operating systems.
macOS
Linux
Windows
- Open the terminal and execute the following command to open the file.
nano ~/.bash_profile
- Add the AKSK information at the end of the file.
export TOS_ACCESS_KEY=AKTPYmI1Z****
export TOS_SECRET_KEY=T1dJM01UU****
- Save the file and exit.
- Execute the following command to apply the configuration.
source ~/.bash_profile
- Execute the following command to verify the configuration.
echo $TOS_ACCESS_KEY
echo $TOS_SECRET_KEY
If the configuration succeeds, the following example will be returned:
AKTPYmI1Z****
T1dJM01UU****
- Open the terminal and run the following command to open the file.
sudo vim /etc/profile
- Add AKSK information at the end of the file.
export TOS_ACCESS_KEY=AKTPYmI1Z****
export TOS_SECRET_KEY=T1dJM01UU****
- Save the file and exit.
- Run the following command to apply the configuration.
source /etc/profile
- Run the following command to verify the configuration.
echo $TOS_ACCESS_KEY
echo $TOS_SECRET_KEY
If the configuration is successful, the following example is returned:
AKTPYmI1Z****
T1dJM01UU****
- open the command line tool and run the following command to add AKSK information.
set TOS_ACCESS_KEY=AKTPYmI1Z****
set TOS_SECRET_KEY=T1dJM01UU****
- run the following command to apply the configuration.
setx TOS_ACCESS_KEY "%TOS_ACCESS_KEY%"
setx TOS_SECRET_KEY "%TOS_SECRET_KEY%"
- restart the command line tool, then run the following command to verify the configuration.
echo %TOS_ACCESS_KEY%
echo %TOS_SECRET_KEY%
if the configuration is successful, the following example will be returned:
AKTPYmI1Z****
T1dJM01UU****
Create TosClient
Initialize with required parameters
The following code initializes using the TOS domain name.
tip
- Endpoint is the publicly accessible domain name for TOS. For supported TOS Endpoint information, see Endpoint.
- AKSK information can be obtained from environment variables. For configuration, see Configure access credentials.
import os
import tos
# Set AKSK via environment variables
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
endpoint = "your endpoint"
region = "your region"
client = tos.TosClientV2(ak, sk, endpoint, region)
Use optional parameters
The following code initializes using optional parameters.
import tos
import os
# ak and sk are security keys in temporary access credentials (AccessKeyId, SecretAccessKey)
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
sts_token = "your security token"
endpoint = "your endpoint"
region = "your region"
# Initialize TosClient using the authentication information in temporary access credentials
client = tos.TosClientV2(ak, sk, endpoint, region,
# Set STS through the optional security_token parameter
security_token=sts_token,
# Set connection timeout (in seconds) through the optional connection_time parameter
connection_time=10,
# Set socket read/write timeout (in seconds) through the optional socket_timeout parameter
socket_timeout=30)
Common configuration scenarios
Configure STS
You can issue temporary access credentials to other users through the STS service. Within the credentials' validity period, users can access your TOS resources without exposing your long-term keys, ensuring the security of your TOS resources.
tip
You can obtain temporary access credentials via the AssumeRole interface of the STS service. These credentials include security keys (AccessKeyId, SecretAccessKey) and a security token (SessionToken). You can then use this information to access the TOS service. For basic operations on setting up the STS service, refer to using STS temporary AK/SK+Token to access Volcengine TOS.
The following code initializes using STS.
import os
import tos
# ak and sk are security keys (AccessKeyId, SecretAccessKey) in temporary access credentials
ak = "your access key"
sk = "your secret key"
# sts_token is the security token (SessionToken) in temporary access credentials
sts_token = "your security token"
endpoint = "your endpoint"
region = "your region"
# Initialize TosClient using authentication information from temporary access credentials
client = tos.TosClientV2(ak, sk, endpoint, region, security_token=sts_token)
Configure timeout mechanism
You can configure network request timeout when initializing TosClient by adding optional parameters. Currently, the TOS Python SDK supports configuring connection timeout connection_time and request timeout socket_timeout.
The following code configures timeout when initializing TosClient.
import tos
import os
# Get AKSK information from environment variables.
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
endpoint = "your endpoint"
region = "your region"
# create a TosClientV2 object and set the connection timeout to 30 seconds and the socket read/write timeout to 60 seconds
# connection_time is the connection timeout, and socket_timeout is the request timeout.
client = tos.TosClientV2(ak, sk, endpoint, region, connection_time=30, socket_timeout=60)
Configure retry policy
Network fluctuations or short-term TOS service exceptions (returning a 500 status code) can cause TOS service call failures. To address this, the Python SDK has an internal retry mechanism. Idempotent operation interfaces retry using an exponential backoff strategy after failure, while non-idempotent interfaces throw the corresponding exception directly. You can add optional parameters to configure the number of retries when initializing TosClient.
import os
import tos
# Obtain AKSK information from environment variables
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
endpoint = "your endpoint"
region = "your region"
# Create a TosClientV2 object and set the timeout retry count to 3. The retry policy uses an exponential backoff strategy with a base time of 100ms: the first retry waits 100ms, the second 200ms, and the third 400ms.
client = tos.TosClientV2(ak, sk, endpoint, region, max_retry_count=3)
Configure data validation
When uploading or downloading objects, TOS returns the object's CRC value. The SDK verifies that the locally calculated CRC matches the CRC returned by TOS to ensure data integrity during upload and download. To ensure data integrity during uploads and downloads, the SDK enables CRC data validation by default. You can disable the CRC validation feature by adding optional parameters when initializing TosClient.
danger
It is strongly recommended not to disable CRC data validation. If this feature is disabled, TOS does not guarantee data integrity during uploads and downloads.
import os
import tos
# Get AKSK information from environment variables
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
endpoint = "your endpoint"
region = "your region"
# Create a TosClientV2 object and disable CRC data verification by setting enable_crc=False
client = tos.TosClientV2(ak, sk, endpoint, region, enable_crc=True)
Configure proxy server
The TOS Python SDK supports accessing TOS services through a proxy server. You can configure the proxy server by adding optional parameters when initializing TosClient.
tip
Currently, only HTTP protocol proxy is supported.
import os
import tos
# Get AKSK information from environment variables
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
endpoint = "your endpoint"
region = "your region"
# Create a TosClientV2 object and configure a proxy using optional parameters proxy_host, proxy_port, proxy_username, and proxy_password
client = tos.TosClientV2(ak, sk, endpoint, region, proxy_host='<your proxy host>', proxy_port=80, proxy_username='<your proxy user name>', proxy_password='<your proxy password>')
Configure connection pool
TOS Python SDK supports connection pool configuration. You can add an optional parameter to set the pool size when initializing TosClient.
tip
The default connection pool size is 1024. It is recommended to use a single TosClient instance for single-process, multi-threaded scenarios.
import os
import tos
# Set AKSK via environment variables
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
endpoint = "your endpoint"
region = "your region"
# Create a TosClientV2 object and set max_connections=1024 to configure the connection pool size to 1024
client = tos.TosClientV2(ak, sk, endpoint, region, max_connections=1024)
Associating a custom domain
TOS Python SDK supports setting custom domains. You can pass a custom domain as the Host when initializing TosClient. After specifying a custom domain, you do not need to pass the Bucket in SDK interfaces.
tip
Before using a custom domain, you need to configure it on the server side. For details, see Associating a custom domain.
import os
import tos
# set AKSK via environment variables
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
# fill in your custom domain and the corresponding region of your Bucket for custom_domain and your region # for example, for North China 2 (Beijing), fill in cn-beijing for your region
custom_domain = "your custom endpoint"
region = "your region"
# create a TosClientV2 object and set is_custom_domain=True to indicate the current domain is a custom domain
# before using a custom domain, configure it on TOS
client = tos.TosClientV2(ak, sk, custom_domain, region, is_custom_domain=True)
Configure ECS password-free login
Passwordless login refers to authentication that does not require specifying a key, but instead uses temporary security credentials provided by the Security Token Service (STS) to verify identity and complete authorization. This method does not require revealing a long-term AccessKey, thus ensuring the security of keys under the account.
TOS Python SDK supports ECS password-free login. You can add the credentials_provider parameter when initializing TosClient.
tip
Before using ECS password-free login, first bind an IAM role to ECS. For details, see Binding IAM Role to ECS, and initialize TosClient using the role name as a parameter.
endpoint = "your endpoint"
region = "your region"
role_name = "your role name"
# Create a TosClientV2 object and set icredentials_provider=tos.EcsCredentialsProvider() to use ECS keyless login
# Create an IAM role and bind it to ECS before using password-free login for ECS
client = tos.TosClientV2(endpoint=endpoint, region=region, credentials_provider=tos.EcsCredentialsProvider(role_name))
Initialization parameter list
The following parameters are supported for initializing TosClient.
Parameter | Type | Required | Example value | Description |
Parameter | Type | Required | Example value | Description |
ak | str | Yes | AKTPYmI1ZGQwMDA0NjlhNGFkMzhjNzM0N2Q0OTQ3ZTV**** | Access Key ID, also known as Key ID. |
sk | str | Yes | T1dJM01UUXpOak0wTVdWak5EUmtOR0poWldJNU1HWmxaV1V5TkdReVl6**** | SecretAccess Key, the private access key. |
endpoint | str | Yes | tos-cn-beijing.volces.com | Access domain name. |
region | str | Yes | cn-beijing | TOS server region. |
security_token | str | No | STSeyJBY2NvdW50SWQiOjIxMDAwMDUyMjQsIklkZW50aXR5VHlwZSI6NCwiSWRlbnRpdHlJZCI6MTE3MjI5NiwiQ2hhbm5lbCI6IlVzZXIiLCJBY2Nlc3NLZXlJZCI6IkFLVFBZbUkxWkdRd01EQTBOamxoTkdGa016aGpOek0wTjJRME9UUTNaVFZqTXpFIiwiU2l**** | Security token in temporary access credentials. |
enable_verify_ssl | bool | No | True | Configure whether to enable SSL certificate verification. |
auto_recognize_content_type | bool | No | True | Automatically identify MIME type. |
connection_timeout | int | No | 10 | Set the connection timeout in seconds. |
max_connections | int | No | 1024 | Set connection pool size. |
enable_crc | bool | No | True | Enable or disable CRC data verification. |
max_retry_count | int | No | 3 | Maximum retry attempts for request failure. |
dns_cache_time | int | No | 5 | DNS cache validity period in minutes. Default is 0. A value less than or equal to 0 means DNS caching is disabled. |
proxy_host | str | No | None | Proxy server host address, currently only supports HTTP protocol. |
proxy_port | int | No | None | Proxy server port number. |
proxy_username | str | No | None | Username for connecting to the proxy server. |
proxy_password | str | No | None | The password used to connect to the proxy server. |
is_custom_domain | bool | No | False | Whether to append the bucket name before the endpoint to access TOS via virtual-hosted style. The values are described as follows:
Default is false. tip Set isCustomDomain to true when accessing TOS through a custom domain. |
high_latency_log_threshold | int | No | 100 | A value greater than 0 indicates high latency logging is enabled, unit: KB. The default is 100. When the total transmission rate of a single request is below this value and the total request duration exceeds 500 milliseconds, a WARN level log is recorded. |
socket_timeout | int | No | 30 | Socket read and write timeout for a single request after connection is established. Unit: seconds. Default is 30 seconds. |
credentials_provider | object | No | None | Supports permanent access keys, temporary access keys, ECS password-free login, environment variable access key retrieval, and other methods via credentials_provider. |
Last updated: 2026.07.31 18:26:50
All rights reserved © Beijing Volcano Engine Technology Co., Ltd2024
