Lake AI Service
Lake AI Service
- Documentation
Lake AI ServiceMetadata managementStorage optimization and governanceSnapshot cleanup
Copy page
Download PDFCopy pageDownload PDFThe AI Data Lake Service (LAS) provides multiple storage optimization features, helping users improve query performance and optimize storage resources through capabilities such as snapshot cleanup. The following is a detailed description and operation guide for the snapshot cleanup feature within storage optimization.
Currently, only tables of the Lance and Iceberg types support the storage optimization - snapshot cleanup feature.
Enable and configure
Log in to LAS console . Select the region at the top left as needed. In the left navigation pane, select Lake management > Metadata. Select the corresponding Database > Table. Navigate to the table list page.
Select the target table, go to Table details > Storage optimizationinterface, enable the Snapshot cleanup feature, and configure the parameters for snapshot cleanup.
Parameters
Configuration description
Resource allocation
Set the number of resource CUs allocated to the current snapshot cleanup task.
Currently, public queue resources are supported for executing storage optimization tasks. After setting resource allocation, the corresponding quota of public queue resources will be occupied when executing storage optimization tasks.Snapshot Retention Policy
Set the retention time range for snapshots. Snapshots that exceed the retention time will be cleaned up during the execution of optimization tasks.
Scheduling
Select whether to enable the periodic scheduling switch.
- Disable periodic scheduling: After the task configuration is completed, you will need to manually trigger task execution.
- Enable periodic scheduling: You also need to configure the periodic scheduling strategy. After the task configuration is completed, tasks will be automatically executed according to the configured scheduling cycle.
Stop Pre Task
Set whether to forcibly terminate preceding optimization tasks. After enabling the switch, forcibly terminate the previous optimization task each time a scheduled task is executed.
Custom configuration
Configure custom parameters involved in the snapshot cleanup task as needed during its execution. For details, see Reference: Custom parameter description below.
Reference: Custom parameter description
Iceberg
The following Iceberg custom parameters are supported for configuration.
Custom parameters | Parameter description |
|---|---|
older_than | Specifies that snapshots earlier than this timestamp can be cleaned up. |
retention_days | The number of days to retain snapshots. |
max_concurrent_deletes | The maximum number of concurrent deletions allowed when deleting files. |
stream_results | Whether to return the list of files to be deleted in a streaming manner to reduce memory usage. |
snapshot_ids | Explicitly specify the list of snapshot IDs to be cleaned up. |
Lance
The following Lance custom parameters are supported for configuration.
Custom parameters | Parameter description |
|---|---|
error_if_tagged_old_versions | Whether to report an error and interrupt when tagged old versions exist within the cleanup scope, to avoid accidentally deleting versions marked for retention. |
After enabling and configuring the snapshot cleanup feature, you can manually trigger the execution of optimization tasks on the snapshot cleanup page, and also view optimization records and results.
