Docs

Torch Object Storage

Vector

PutVectors（TOS Vectors Python SDK）

The put_vectors interface writes vector data to a specified collection. This article explains how to perform batch vector writing using the put_vectors interface of the TOS Vectors Python SDK.

Notes

You need to fill in the region and access domain in the example code according to your actual situation. For details, see Region and Access Domain (Endpoint).

To prevent key leakage, the example code reads the Access Key ID and Secret Access Key from environment variables. Therefore, before using the SDK, you need to configure long-term access credentials. For details, see Configure Long-Term Access Credentials.

TOS Vectors Python SDK supports temporary access credentials; it also supports configuring timeout, retry policies, and other mechanisms. You can flexibly set these according to business needs to enhance application security. For details, see Common Configuration Scenarios.

TOS Vectors supports float32 precision for vectors. Passing vectors with higher precision may cause precision loss.

Permissions required

The main account has full permissions by default, but IAM users have no permissions by default. If you need to use a sub-account to perform the put_vectors operation, the main account must grant the following permissions to the sub-account through the IAM policy and bucket policy:

tosvectors:PutVectors

Code example

Batch insert vectors into the vector index

# Batch insert vectors into the vector index.​
import os​
import tos​
​
# Retrieve AK and SK information from environment variables.​
ak = os.getenv('TOS_ACCESS_KEY')​
sk = os.getenv('TOS_SECRET_KEY')​
# Fill in your endpoint and your region with the endpoint corresponding to the bucket's region. # For example, for China (Beijing), set your endpoint to https://tosvectors-cn-beijing.volces.com and your region to cn-beijing.​
endpoint = 'Provide your endpoint'​
region = 'Provide your region'​
account_id = os.getenv('TOS_ACCOUNT_ID')​
vector_bucket_name = 'Provide your bucket name'​
​
try:​
    # Create a VectorClient object; all vector operations are handled through VectorClient.​
    client = tos.VectorClient(ak, sk, endpoint, region)​
    ​
    # Prepare vector data - Create 3 128-dimensional float32 vectors​
    vectors = []​
    for i in range(3):​
        # Create vector data, with each vector having 128 dimensions​
        vector_data = tos.models2.VectorData(​
            float32=[float(x) for x in range(128)]  # 128-dimensional vector data​
        )​
        ​
        # Create a vector object​
        vector = tos.models2.Vector(​
            key=f'vector-{i}',  # Vector unique key​
            data=vector_data    # Vector data​
        )​
        vectors.append(vector)​
    ​
    # Specify index name​
    index_name = 'test'​
    ​
    # Batch insert vectors into the vector index.​
    result = client.put_vectors(​
        vector_bucket_name=vector_bucket_name,​
        account_id=account_id,​
        index_name=index_name,​
        vectors=vectors​
    )​
    ​
    # Print the operation result.​
    print('Batch insert vectors successful')​
    print('Request ID:', result.request_id)​
    print('Status Code:', result.status_code)​
    print('Number of successfully inserted vectors:', len(vectors))​
    ​
except tos.exceptions.TosClientError as e:​
    # The operation failed and a client exception was caught. This is usually due to invalid request parameters or a network exception.​
    print('fail with client error, message:{}, cause: {}'.format(e.message, e.cause))​
except tos.exceptions.TosServerError as e:​
    # The operation failed and a server exception occurred. See the response for the error details.​
    print('fail with server error, code: {}'.format(e.code))​
    # Use the request ID for troubleshooting purposes. We strongly recommend including the request ID in the logs.​
    print('error with request id: {}'.format(e.request_id))​
    print('error with message: {}'.format(e.message))​
    print('error with http code: {}'.format(e.status_code))​
    print('error with ec: {}'.format(e.ec))​
    print('error with request url: {}'.format(e.request_url))​
except Exception as e:​
    print('fail with unknown error: {}'.format(e))​

Batch insert vectors with metadata into the vector index

# Batch insert vectors with metadata into the vector index.​
import os​
import tos​
import time​
​
# Retrieve AK and SK information from environment variables.​
ak = os.getenv('TOS_ACCESS_KEY')​
sk = os.getenv('TOS_SECRET_KEY')​
# Fill in your endpoint and your region with the endpoint corresponding to the bucket's region. # For example, for China (Beijing), set your endpoint to https://tosvectors-cn-beijing.volces.com and your region to cn-beijing.​
endpoint = 'Provide your endpoint'​
region = 'Provide your region'​
account_id = os.getenv('TOS_ACCOUNT_ID')​
vector_bucket_name = 'Provide your bucket name'​
​
try:​
    # Create a VectorClient object; all vector operations are handled through VectorClient.​
    client = tos.VectorClient(ak, sk, endpoint, region)​
    ​
    # Prepare vector data - Create vectors with metadata​
    vectors = []​
    categories = ['electronics', 'clothing', 'books', 'sports', 'home']​
    ​
    for i in range(5):​
        # Create vector data, with each vector having 256 dimensions​
        vector_data = tos.models2.VectorData(​
            float32=[float(i * 0.1 + x * 0.01) for x in range(128)]  # 128-dimensional vector data​
        )​
        ​
        # Create a vector object with metadata​
        vector = tos.models2.Vector(​
            key=f'product-vector-{i}',  # Vector unique Key​
            data=vector_data,            # Vector data​
            metadata={                   # Vector metadata​
                'product_id': f'prod_{i:06d}',​
                'category': categories[i % len(categories)],​
                'name': f'Product {i}',​
                'price': round(10.0 + i * 5.5, 2),​
                'timestamp': str(int(time.time() * 1000)),​
                'description': f'This is product {i} in {categories[i % len(categories)]} category'​
            }​
        )​
        vectors.append(vector)​
    ​
    # Specify index name.​
    index_name = 'test'​
    ​
    # Batch insert vectors with metadata into the vector index.​
    result = client.put_vectors(​
        vector_bucket_name=vector_bucket_name,​
        account_id=account_id,​
        index_name=index_name,​
        vectors=vectors​
    )​
    ​
    # 打印操作结果​
    print('Bulk insert vectors with metadata successfully')​
    print('Request ID:', result.request_id)​
    print('Status Code:', result.status_code)​
    print('Number of successfully inserted vectors:', len(vectors))​
    print('\n Inserted vector information:')​
    for i, vector in enumerate(vectors):​
        print(f'  向量 {i+1}: key={vector.key}, category={vector.metadata["category"]}, price=${vector.metadata["price"]}')​
    ​
except tos.exceptions.TosClientError as e:​
    # The operation failed and a client exception was caught. This is usually due to invalid request parameters or a network exception.​
    print('fail with client error, message:{}, cause: {}'.format(e.message, e.cause))​
except tos.exceptions.TosServerError as e:​
    # The operation failed and a server exception occurred. See the response for the error details.​
    print('fail with server error, code: {}'.format(e.code))​
    # Use the request ID for troubleshooting purposes. We strongly recommend including the request ID in the logs.​
    print('error with request id: {}'.format(e.request_id))​
    print('error with message: {}'.format(e.message))​
    print('error with http code: {}'.format(e.status_code))​
    print('error with ec: {}'.format(e.ec))​
    print('error with request url: {}'.format(e.request_url))​
except Exception as e:​
    print('fail with unknown error: {}'.format(e))​