Shard upload is suitable for uploading larger objects. The principle is to split the data source into multiple shardings and upload them separately, and then merge them to generate the final object.

Upload steps

The steps to use the TOS shard upload interface are as follows:

Create a shard upload task through TosClient- > createMultipartUpload and get the UploadID.

Upload multiple shardings via TosClient- > uploadPart.

warning

The size of a single sharding cannot exceed 5GiB, and a maximum of 10,000 shardings are supported.

Except for the last sharding, the size of each sharding cannot be less than 4MiB, and the verification of the sharding size at the TOS server level will be performed when merging shardings.

Each sharding is uniquely identified by a sharding number (PartNumber), which is valid between 1 and 10,000. For the same sharding number, the previously uploaded sharding data can be overwritten by repeated uploads.

The final object is generated by combining sharding with TosClient- > completeMultipartUpload.

Sample code

The following example shows the complete code for local large file upload via sharded upload:

<?php​
​
// Assuming the use of composer for installation.​
require_once __DIR__ . '/vendor/autoload.php';​
​
use Tos\TosClient;​
use Tos\Exception\TosClientException;​
use Tos\Exception\TosServerException;​
use Tos\Model\CompleteMultipartUploadInput;​
use Tos\Model\CreateMultipartUploadInput;​
use Tos\Model\Enum;​
use Tos\Model\UploadedPart;​
use Tos\Model\UploadPartInput;​
use GuzzleHttp\Psr7\Utils;​
​
$file = null;​
try {​
    $client = new TosClient([​
        'region' => 'your region',​
        'endpoint' => 'your endpoint',​
        // Get access key from environment variables.​
        'ak' => getenv('TOS_ACCESS_KEY'),​
        'sk' => getenv('TOS_SECRET_KEY'),​
    ]);​
    ​
    $bucket = 'bucket-test';​
    $key = 'key-test';​
    ​
    // Step 1: Create a multipart upload task​
    $input = new CreateMultipartUploadInput($bucket, $key);​
    // Set object ACL​
    $input->setACL(Enum::ACLPublicRead);​
    // Set object StorageClass​
    $input->setStorageClass(Enum::StorageClassStandard);​
    // Set object custom metadata​
    $input->setMeta(['aaa' => 'bbb', 'Chinese Key' => 'Chinese Value']);​
    // Set object Content-Type​
    $input->setContentType('text/plain');​
    $output = $client->createMultipartUpload($input);​
    echo $output->getRequestId() . PHP_EOL;​
    echo 'createMultipartUpload succeed' . PHP_EOL;​
​
    // Get UploadID​
    $uploadId = $output->getUploadID();​
​
    // Step 2: Upload multiple chunks​
    // Assuming the local big file path is local_big_file_path​
    $localBigFilePath = 'local_big_file_path';​
    // Assuming the big file is split into 20MB chunks​
    $partSize = intval(20 * 1024 * 1024);​
    $fileSize = filesize($localBigFilePath);​
​
    $partCount = intval($fileSize / $partSize);​
    if (($lastPartSize = $fileSize % $partSize) !== 0) {​
        $partCount++;​
    } else {​
        $lastPartSize = $partSize;​
    }​
​
    $parts = [];​
    for ($i = 0; $i < $partCount; $i++) {​
        $partNumber = $i + 1;​
        $file = fopen($localBigFilePath, 'r');​
        // Set the starting position of the currently uploaded file.​
        fseek($file, $partSize * $i, 0);​
        $input = new UploadPartInput($bucket, $key, $uploadId, $partNumber);​
        if ($i === $partCount - 1) {​
            // Handling the last fragment.​
            $input->setContentLength($lastPartSize);​
        } else {​
            $input->setContentLength($partSize);​
        }​
        $input->setContent($file);​
        $output = $client->uploadPart($input);​
        echo $output->getRequestId() . PHP_EOL;​
        echo sprintf('upload part %d succeed', $partNumber) . PHP_EOL;​
        if (is_resource($file)) {​
            fclose($file);​
        }​
        // Collect all the fragments.​
        $parts[] = new UploadedPart($partNumber, $output->getETag());​
    }​
​
    // Step Three: Merge the fragments.​
    $input = new CompleteMultipartUploadInput($bucket, $key, $uploadId, $parts);​
    $output = $client->completeMultipartUpload($input);​
    echo $output->getRequestId() . PHP_EOL;​
    echo 'completeMultipartUpload succeed' . PHP_EOL;​
​
} catch (TosClientException $ex) {​
    echo $ex->getMessage() . PHP_EOL;​
} catch (TosServerException $ex) {​
    echo $ex->getRequestId() . PHP_EOL;​
    echo $ex->getStatusCode() . PHP_EOL;​
    echo $ex->getErrorCode() . PHP_EOL;​
} finally {​
    if (is_resource($file)) {​
        fclose($file);​
    }​
}​

Last updated: 2025.09.23 14:41:21

Public security network record in Beijing: No. 11010802032137