Write: Use open(..., 'wb') to create a writable file handle. Then, continuously write multiple GB-scale objects and compare the end-to-end write throughput.

Read: Use open(..., 'rb') to create a readable file handle. Then, continuously read multiple GB-scale objects and compare the end-to-end read throughput.

Test data:

Concurrency: 1 — Write/read N objects totaling approximately 10 GB.

Concurrency: 32 — Write/read N objects totaling approximately 10 GB.

Test environment:

Volcano Engine Elastic Compute Service instance: ecs.g3ine.24xlarge, with 96 vCPUs, 384 GiB of memory, and 144 Gbps of intranet bandwidth.

Concurrent write throughput for large objects

The table below shows the end-to-end write throughput (MB/s) for large objects when using different tools and at different concurrency levels.

Tool

Concurrency

Write throughput (MB/s)

Tool	Concurrency	Write throughput (MB/s)
tosfsspec	1	1933.25
tosfsspec		32	15425.33
tosfs tip This is an unofficial legacy implementation for TOS.	1	66.03
		32	2054.13
s3fs	1	67.53
s3fs		32	2121.28

Concurrent read throughput for large objects

The table below shows the end-to-end read throughput (MB/s) for large objects when using different tools and at different concurrency levels.

Tool

Concurrency

Read throughput (MB/s)

Tool	Concurrency	Read throughput (MB/s)
tosfsspec	1	1702.18
tosfsspec		32	10102.15
tosfs tip This is an unofficial legacy implementation for TOS.	1	147.84
		32	2189.14
s3fs	1	427.41
s3fs		32	4586.67

Summary

tosfsspec delivers higher end-to-end read and write throughput, providing superior overall performance to other fsspec implementations. It is ideal for scenarios requiring high-bandwidth access to TOS, such as importing/exporting data warehouses, archiving large volumes of logs and preparing machine learning training data.

Last updated: 2026.03.31 18:14:37

Public security network record in Beijing: No. 11010802032137