LAS provides the LAS Catalog metadata backup and recovery tool lf-exporter, which you can use to perform same-region and cross-region disaster recovery for metadata (catalog, database, table, partition, function). This document introduces the key points for using the lf-exporter tool and recommendations for disaster recovery solutions.

The LAS Catalog metadata backup and recovery tool lf-exporter runs as an VolcanoEngineEMR Serverless Spark job. After obtaining the toolkit, configure the YAML configuration file and submit an EMR Serverless Spark job to invoke the lf-exporter tool, enabling LAS Catalog metadata backup and recovery.
In the YAML configuration file, the mode parameter specifies the operation mode for LAS Catalog metadata backup and recovery, supporting backup and recovery of metadata in the following four scenarios.

When using the lf-exporter tool, after completing the necessary preparations, the general workflow for each scenario is: configure the YAML configuration file > run the EMR Serverless job to invoke the tool > verify the results.
The following section provides detailed instructions for using the lf-exporter tool in different scenarios.

Step 1: Obtain the lf-exporter tool #

You need to first obtain the lf-exporter tool and upload it to TOS, so that it can be selected when running EMR Serverless Spark jobs later.

wget https://lasformation-cn-beijing.tos-cn-beijing.volces.com/las-exporter/application.jar

Step 2: Prepare the YAML configuration file #

Prepare the YAML configuration file based on your usage scenario. The following are example YAML configuration files for each scenario.

• When actually using the tool, replace ${AK}, ${SK}, TOS paths, and other related parameters as appropriate for your environment.
• For the complete list of configuration parameters and their descriptions, see the following section: Reference: Configuration parameter list.

mode: las_to_tos
lasClientInfo:
  akMode: MANUAL
  accessKeyId: "${AK}"
  accessKeySecret: "${SK}"
  endPoint: thrift://lakeformation.las.cn-beijing.ivolces.com:48869
  regionId: cn-beijing
  # Required for STS mode
  securityToken: "${STS_SESSION_TOKEN}"
  identityId: "${ACCOUNT_ID}-${ROLE_ID}"
runOptions:
  includeCatalogPrefixs: []
  includeDatabasePrefixs: []
  includeTablePrefixs: []
  objectTypes: [catalog, database, table, partition, function]
  permissionDeniedStrategy: skip
  outputBaseDir: tos://backup-bucket/las_exporter/
  auditOutputDir: tos://backup-bucket/las_exporter_audit/

mode: tos_to_las
lasClientInfo:
  akMode: MANUAL
  accessKeyId: "${AK}"
  accessKeySecret: "${SK}"
  endPoint: thrift://lakeformation.las.cn-beijing.ivolces.com:48869
  regionId: cn-beijing
  # Required for STS mode
  securityToken: "${STS_SESSION_TOKEN}"
  identityId: "${ACCOUNT_ID}-${ROLE_ID}"
runOptions:
  includeCatalogPrefixs: []
  includeDatabasePrefixs: []
  includeTablePrefixs: []
  batchSize: 1000
  sparkTaskBatchSize: 200000
  objectTypes: [catalog, database, table, partition, function]
  permissionDeniedStrategy: skip
  inputBaseDir: tos://backup-bucket/las_exporter/2026-04-13/
  auditOutputDir: tos://backup-bucket/las_exporter_audit/
  locationMappings:
    - source: tos://old-bucket/
      target: tos://new-bucket/

mode: las_to_hive
lasClientInfo:
  akMode: AUTO
  endPoint: thrift://lakeformation.las.cn-beijing.ivolces.com:48869
  regionId: cn-beijing
hmsClientInfo:
  hiveConfPath: /etc/emr/hive/conf/hive-site.xml
runOptions:
  includeCatalogPrefixs: [las_exporter]
  includeDatabasePrefixs: [las_exporter]
  includeTablePrefixs: [exporter_table_1000]
  batchSize: 1000
  objectTypes:
    - catalog
    - database
    - table
    - partition
    - function

mode: tos_to_hive
lasClientInfo:
  akMode: AUTO
  endPoint: thrift://lakeformation.las.cn-beijing.ivolces.com:48869
  regionId: cn-beijing
hmsClientInfo:
  hiveConfPath: /etc/emr/hive/conf/hive-site.xml
runOptions:
  includeCatalogPrefixs: [las_exporter]
  includeDatabasePrefixs: [las_exporter]
  objectTypes:
    - catalog
    - database
    - table
    - partition
    - function
  outputBaseDir: xxx
  inputBaseDir: tos://exporter_las/2025-02-19

Step 3: Upload the toolkit and configuration file to TOS #

Upload the toolkit and configuration file to the TOS Bucket, so that they can be referenced when submitting and running EMR Serverless Spark jobs later. For details on uploading files to a TOS Bucket, see: Upload files .

Step 4: Submit a Serverless Spark job #

1. Log in and go to the EMR console . On the job management page, submit a Spark JAR job. For detailed instructions on submitting the job, see the EMR official documentation: Submit Spark JAR jobs .

2. In the job content, configure the job details to invoke the lf-exporter tool. Below is a SparkJar JSON task parameter template. You can refer to this template and modify the parameters as needed based on your actual situation.

   {
       "JobType": "SparkJar",
       "VolcanoQueue": null,
       "StorageMounts": [],
       "MainClass": "bytedance.olap.las.Exporter",
       "MainApplicationFile": "tos://las_exporter/application.jar",
       "Jars": [],
       "Files": [
           "tos://las_exporter/tos_to_las.yaml"
       ],
       "PyFiles": null,
       "Archives": [],
       "SparkConf": "spark.sql.adaptive.enabled=false,spark.driver.cores=1,spark.driver.memory=4G,spark.executor.cores=2,spark.executor.memory=4G,spark.executor.instances=2,spark.sql.shuffle.partitions=200,serverless.spark.access.key=AKLTN******,serverless.spark.secret.key=TnpBe******",
       "MainArguments": "tos://las_exporter/tos_to_las.yaml"
   }
   

   Among them:

• MainApplicationFile: Set this to the TOS path of the lf-exporter tool JAR package after uploading it to the TOS Bucket.
• Files: Set this to the TOS path of the YAML configuration file for the corresponding task after uploading it to the TOS Bucket.
• MainArguments: Set this to the TOS path of the YAML configuration file for the corresponding task after uploading it to the TOS Bucket.
• SparkConf：
  • serverless.spark.access.key and serverless.spark.secret.key: Set these to the AK and SK with TOS and Catalog access permissions obtained during preparation.

Step 5: Confirm the backup/restore result #

• After a successful backup:
  • In outputBaseDir, you can see backup directories partitioned by date, which include JSON files for each object type (for example, catalogs/, databases/, tables/, partitions/, functions/). For details on the organization of the output, see Reference: Organization of backup artifacts below for details.
  • If auditOutputDir is configured, an audit summary will also be output to help confirm whether the backup was fully successful.
• After restore is complete:
  • You can verify that the catalog, db, table, partition, and function are visible in the LAS Catalog console or by querying from the engine side.
  • By default, restore uses an idempotent strategy of "skip if exists", which is suitable for repeated executions and failure retries.

Disaster recovery solution recommendations #

• When planning a disaster recovery solution based on the lf-exporter tool, note that the lf-exporter tool currently does not copy data files. To ensure data availability for cross-region disaster recovery, an additional data file replication mechanism is required. Common approaches include cross-region replication of object storage TOS or using directory-level replication tools in a controlled compute environment.
  
  
• The key inputs for data file disaster recovery come from Table.sd.location, Partition.sd.location, and Function.resourceUris[].uri in the metadata backup. It is recommended to first extract these location prefixes from the backup JSON, calculate the target prefixes using locationMappings, then perform consistency checks on the target prefixes, and use the check results as a prerequisite for metadata restoration.

Disaster recovery configuration #

For the cold backup scenario across regions, this section provides an example configuration process using TOS cross-region replication.

1. Refer to the general usage process of the lf-exporter tool. For the las_to_tos task scenario, configure it to back up LAS Catalog metadata to a TOS Bucket in the same region.
2. Use the cross-region replication feature of TOS to replicate metadata backed up to TOS across regions to another Bucket in a different region. For example, replicate the contents of a Bucket in the North China region to a Bucket in the East China region. For configuration details, see: TOS Cross-Region Replication Reference Documentation .
3. After creating cross-region replication rules and completing synchronization, you can check the replication results in the remote Bucket.

Exception recovery #

When the metadata in the primary service region encounters an exception, you can restore the metadata backed up to TOS to LAS. The core process is as follows.

1. You can create a new TOS Bucket in the primary service region for restoring metadata storage.
2. You can use the cross-region replication feature of TOS to replicate the metadata from the remote Bucket back to the recovery Bucket in the same region. For configuration details, see: TOS Cross-Region Replication Reference Documentation .

   warning

   The LAS Catalog metadata backed up to TOS is serialized as JSON and stored in date-partitioned directories. For details, see Reference: Organization of backup artifacts below. When performing a restore, you must specify the required backup date directory.

3. Refer to the general usage process of the lf-exporter tool. Based on the configuration for the tos_to_las task scenario, restore the metadata in the TOS Bucket to the LAS Catalog in the same region.
4. After the job is completed, go to LAS to view the recovery results.

In las_to_tos mode, metadata objects are serialized as JSON and stored in the date-partitioned directories under runOptions.outputBaseDir according to object type. In recovery mode (tos_to_las / tos_to_hive), runOptions.inputBaseDir needs to point to the date directory of a specific backup.

The following table lists the parameter descriptions used in the backup configuration.