Docs

Big Data File Storage

Appendix

Cache acceleration CLI commands

When cache acceleration is enabled for a CloudFS instance, CLI commands are supported. This article describes how to use commonly used CLI commands in cache acceleration scenarios.

Deploy CLI

Machine learning platform [development machine] and [custom training] have integrated CLI, so deployment is not required and can be used directly.

Run the following command to download version 1.5.6.65 of the CloudfsSDK package.

https://cloudfs.tos-cn-beijing.volces.com/sdk/prod/cloudfs-client-1.5.6.65.tar.gz​

Run the following command to extract the CloudfsSDK package.

tar -zxvf cloudfs-client-1.5.6.65.tar.gz

Configure environment variables.

export LD_LIBRARY_PATH=${CLOUDFSSDK_HOME}/lib/

CLI configuration file

Machine learning platform [development machine] and [custom training] have integrated CLI, so configuration is not required and can be used directly.

Create and configure the cloudfs.conf configuration file in any directory.

vim ${CLOUDFSSDK_HOME}/conf/cloudfs-fuse.conf

Edit the configuration file cloudfs-fuse.conf.

# CloudFS Client Configurations​
​
# CloudFS endpoint, e.g. cfs://xxx.cfs-cn-beijing.ivolces.com​
cfs.filesystem.uri=​
cfs.filesystem.fs-mode=ACC​
​
# IAM Auth info​
cfs.access.key=​
cfs.secret.key=​
​
# Required settings for ACC mode​
cfs.filesystem.ns-id=​
cfs.filesystem.ufs-path=​

Parameter	Description
cfs.filesystem.uri	The mount target of the file storage instance. When setting, you need to add the prefix cfs:// before the mount target. For information on how to obtain it, see View instance details.
cfs.access.key	The Access Key ID of the account used when creating a file storage instance. For information on how to obtain it, see Key management.
cfs.secret.key	The Secret Access Key of the account used when creating a file storage instance. For information on how to obtain it, see Key management.
cfs.filesystem.ns-id	The Namespace ID of the storage bucket mounted to the file storage instance (that is, the TosNamespace ID). For information on how to obtain it, see View instance details.
cfs.filesystem.ufs-path	The complete storage address of the storage bucket mounted to the file storage instance, in the format tos://<bucket name>/<file storage prefix>. Example: tos://bucket/prefix/.

Execution example

Run the following command to obtain the file list.

Machine learning platform [development machine] and [custom training] can execute the command

cfs-cli ls tos://bucket/object

Self-deployment installation method

${CLOUDFSSDK_HOME}/bin/cfs ls tos://bucket/object --config-file ${CLOUDFSSDK_HOME}/conf/cloudfs-fuse.conf​

Command list

The CLI lists files, allowing CloudFS to synchronize metadata from TOS. For the listed path, enter the TOS path to be executed.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs ls [OPTION] <path>​
    ​
DESCRIPTION​
    -r, --recursive​
            Recursively list subdirectories.​
    -s, --skipSync​
            Skip metadata synchronization.​
        --persistStatus​
            Display the persistence status of the file.​
        --depth​
            Can be used in conjunction with "-R", indicating the depth of list path. "0" represents only the current path of list, "1" represents current path of list and its subpath, and so on. The defaul value is 32767.​
        --concurrency​
            Recursively list the maximum concurrency of subdirectories. The default value is 100.​
        --showPin​
            Display wether the file is pinned.​
        --config-file​
            Configuration file path.​

If no list result is printed after 30 seconds, the total number of files listed so far will be printed to indicate progress. Examples:

1327202 paths have been listed as of Dec 21 10:37:06

load

Load metadata or data from the underlying file system into the CloudFS cache.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs load [OPTION] <path>​
    After the command is run, JOB ID is returned. This JOB ID is used to query task status.​
    ​
DESCRIPTION​
    -d, --data​
            Load data from the underlying file system into CloudFS cache.​
    -m, --metadata​
            Load metadata from the underlying file system into CloudFS cache.​
    -r, --recursive​
           Recursively load data or metadata. ​
        --replica​
            Number of copies of loaded data (default to 1).​
        --config-file​
            Configuration file path.​

free

Release data or metadata in CloudFS (this does not affect data in the underlying file system).

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs free [OPTION] <path>​
    After the command is run, JOB ID is returned. This JOB ID is used to query task status.​
    ​
DESCRIPTION​
    -r, --recursive​
          Recursively release data or metadata ​
        --config-file​
          Configure file path​

lookup

Query task status.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs lookup <job ID>​
 ​
DESCRIPTION  ​
        --config-file​
            Configure file path.​

cancel

Cancel task.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs cancel <job ID>​
​
DESCRIPTION​
        --config-file​
            Configure file path.​

Display cache usage statistics for a path in CloudFS.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs du [OPTION] <path>​
    ​
DESCRIPTION​
    -h, --human-readable      ​
    -s, --summarize​
           Summarize and display the specified path ​
        --config-file​
           Configure file path.​

setPolicy

Set the read/write policy for a path.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs setPolicy [OPTION] <path>​
    ​
DESCRIPTION​
    -u, --uploadIntervalMs​
        Whether to write back to the underlying storage. Currently, only 0 (immediate write back) and -1 (no write back) are supported.​
    --config-file​
        Configure file path.​

getPolicy

Obtain the read/write policy for all paths.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs getPolicy [OPTION] <path>​
    ​
DESCRIPTION​
    --config-file​
        Configure file path.​

listPolicy

Obtain the read/write policy for all paths in the file system instance.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs listPolicy [OPTION]​
    ​
DESCRIPTION​
    --config-file​
        Configure file path.​

removePolicy

Remove the read/write policy for a path.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs removePolicy [OPTION] <path>​
    ​
DESCRIPTION​
    -r, --replicaPolicy​
        Delete replica policy.​
    -u, --uploadPolicy​
        Delete write back policy.​
    --config-file​
        Configure file path.​

Pin file data in the CloudFS cache to prevent the data from being evicted.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs pin [OPTION] <path>​
    ​
DESCRIPTION​
    --recursive​
        Choose whther to execute recursively. Recursive mode will mark all subpaths as PINned, while non-recursive mode only marks the requested path itself. Both modes will mark incremental paths as PINned.​
    --config-file​
        Configure file path.​

unpin

Unpin the pinned path. After unpinning, the data in the cache will be automatically evicted according to the algorithm.

SYNOPSIS​
    ${CLOUDFSSDK_HOME}/bin/cfs unpin [OPTION] <path>​
    ​
DESCRIPTION​
    --recursive​
        Whether to excute recursion.​
    --config-file​
        Configure file path.​

Last updated: 2026.02.02 11:22:42