Docs

Torch Object Storage

Performance optimization

Benchmarking TOS performance using COSBench

COSBench (Cloud Object Storage Benchmark) is a benchmarking tool developed by the Intel team using Java to measure the performance of object storage services. You can use COSBench to conduct stress tests on the read/write performance of TOS. This will enable you to benchmark its performance and ensure business stability.

Operating environment

We recommend running COSBench on CentOS 7.6 or a later version. Using other operating environments may result in unexpected issues.

Using COSBench

Download the COSBench installation package for TOS and extract it to your local device.

tip

COSBench cannot be used directly to test TOS performance. First, you need to modify COSBench‘s source code. Then, you need to build a COSBench adapter for TOS from the modified source code. For instructions, see Appendix: Building a COSBench adapter for TOS.

Run this command to install COSBench dependencies.

sudo yum install nmap-ncat java curl java-1.8.0-openjdk-devel -y

Configure the s3-config-sample.xml file in the COSBench adapter to create a test workload.

Code example

Stages

Parameters

Configure the cosbench-start.sh file and add the following parameters to adjust the buffer size and disable MD5. The buffer size should be at least as large as the size of the test objects. Adjust this value based on your needs.

-Dcom.amazonaws.sdk.s3.defaultStreamBufferSize=104857600 -Dcom.amazonaws.services.s3.disableGetObjectMD5Validation=true -Dcom.amazonaws.services.s3.disablePutObjectMD5Validation=true

Run this command to start the COSBench service.

sudo bash start-all.sh

Differences in versions of the Nmap tool on operating systems such as Debian may cause the nc command's detection operation to fail, which prevents COSBench from starting. If you encounter this issue, you can fix it by adding the -q 1 parameter to the nc command in the cosbench-start.sh file.

Run this command to submit the test workload.

sudo bash cli.sh submit conf/s3-config-sample.xml

To check the test status, open http://ip:19088/controller/index.html in your browser (replace "ip" with the test server's IP address).

After completing your test, run this command to stop the COSBench service.

sudo bash stop-all.sh

Appendix: Building a COSBench adapter for TOS

COSBench cannot be used directly to test TOS performance. First, you need to modify COSBench's source code. Then, you need to build a COSBench adapter for TOS from the modified source code.

Environment

In this section, we will demonstrate how to build a COSBench adapter for TOS using the following environment.

Operating system: macOS x86_64

IDE version: Eclipse 4.15

JDK version: 1.8

Steps

Download the source code package for COSBench from GitHub and extract it to your local device.

Import COSBench‘s source code into Eclipse.

In Eclipse, select File > Import... > Existing Projects into Workspace.

In the Import Projects dialog box, select the dev directory from the source code and click Finish.

Select Eclipse > Settings... > Plug-in Development > Target Platform.

In the Target Platform dialog box, click Add...

In the Target Content dialog box, enter a name for your platform in the Name field.

Select Add... > Directory. Then, add the main, osgi, and osgi/libs folders from the dist directory of the source code, and click Finish.

In the Target Platform dialog box, select the platform you just created and click Apply and Close.

Configure the S3Storage.java file.

Locate the S3Storage.java file in the cosbench-s3 folder.

In the S3Storage.java file, modify the value of clientConf.withSignerOverride to AWSS3V4SignerType to use AWS Signature Version 4. Example:

clientConf.withSignerOverride("AWSS3V4SignerType");

In the S3Storage.java file, modify the value of client.setS3ClientOptions by adding disableChunkedEncoding to disable chunked encoding. Example:

client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(parms.getBoolean(PATH_STYLE_ACCESS_KEY)).disableChunkedEncoding());

In the S3Storage.java file, modify the value of client.putObject to enable uploading objects using BufferedInputStream. Example:

client.putObject(container, object, new BufferedInputStream(data), metadata);

Generate the COSBench adapter bundle.

In Eclipse, select File > Export... > Plug-in Development > Deployable plugins and fragments.

In the Deployable plug-ins and fragments dialog box, click Select All. Then, set Directory to the dist/osgi folder in the source code.

Click the Options tab, and select Use class files compiled in the workspace. End the process by clicking Finish.

Run this command from the source code directory to package the COSBench adapter bundle.

sh pack.sh cosbench-tos

Run this command to zip the COSBench adapter bundle.

zip -r -X cosbench-tos.zip cosbench-tos

Last updated: 2026.08.25 10:39:45

Stage	Description
init	Initializes bucket resources. This stage creates test buckets.
prepare	Writes the data that will be used for the test into buckets. Specify the size and number of objects to be written for reading in the main stage. This stage is not required if the main stage does not include read operations.
main	Defines the behavior of the test by specifying the duration of the test and the type of operation.
cleanup	Deletes the objects created during the test.
dispose	Deletes the test buckets.

Parameter	Description
accesskey	Specifies the access key ID of your Volcano Engine account. tip For instructions on how to view the access key ID and secret access key, see Viewing the Access Key ID and Secret Access Key
secretkey	Specifies the secret access key of your Volcano Engine account.
endpoint	Specifies the S3 endpoint URL. For more information, see Regions and endpoints.
cprefix	Specifies the bucket prefix.
csuffix	Specifies the bucket suffix. This parameter can be null.
containers	Specifies the number of buckets and their naming rules. For example, setting containers=r(1,2) and cprefix=examplebucket creates two buckets, one named examplebucket1 and the other named examplebucket2.
objects	Specifies the number of objects to be written per bucket and their naming rules.
sizes	Specifies the object size.
ratio	Specifies the proportion of different operation types in the test. For example, setting the write ratio to 80 and the read ratio to 20 means write operations will make up 80% of total requests, while read operations will make up the remaining 20%.
runtime	Specifies how many seconds the work will execute for.
workers	Specifies the number of workers to conduct the work in parallel.