Docs

ByteHouse Cloud Data Warehouse Version

Manage tenant

Preload

This article describes the usage process, function, and procedure for creating, viewing, editing, and deleting cache preload tasks.

Introduction

The cache preload (Preload) function is a data cache optimization mechanism in ByteHouse Cloud Data Warehouse, designed to improve query performance. It preloads data from remote object storage into the local cloud disk of the compute group. Cache preload consists of three steps:

Data prefetch: According to the cache strategy, data is loaded into the cache. ByteHouse supports three strategies: offline strategy, real-time strategy, and one-time strategy.
- Offline strategy: Periodically refreshes existing data into the cache.
- Real-time strategy: Automatically retrieves the latest written data and loads it into the cache.
- One-time strategy: Loads data within a specified date range into the cache.
Cache management: ByteHouse Cloud Data Warehouse monitors and manages cached data. When cache space is insufficient, it automatically evicts some less frequently used data based on usage and importance, freeing up space for new data that needs to be cached.
- Offline cache strategy: By default, data is cached according to the TTL (Time To Live) you set when creating the cache preload task. If the size of data within the TTL is greater than the cache space, the LRU (Least Recently Used) eviction strategy is applied.
- Real-time cache strategy: By default, data follows the LRU (Least Recently Used) eviction strategy.
Query acceleration: When the user executes an SQL query, the data is already cached on the local cloud disk of the compute group, allowing for direct and fast data reads without retrieving it from object storage. This greatly reduces data read time, improves query performance, and enables query results to be returned to the user more quickly.

Usage process

From a business scenario perspective, cache preload and the compute group's local disk (Use the virtual warehouse local disk) are generally used together to ensure sufficient cache space is available to accommodate data when loading it into the cache. The overall cache loading usage process is as follows:

Use cases

Industry	Business scenarios	Scenario and function applicability analysis
Internet and e-commerce	Real-time monitoring of user behavior, sales representative trends, promotional activity effectiveness, and more, requiring real-time generation of analysis reports	Real-time data dashboard: By caching frequently accessed sales representative data in real time (such as transaction records from the past 30 days), the timeliness of BI report analysis is compressed from hourly to second-level. User behavior analysis: Pre-cache user behavior logs (such as click and add-to-cart data), support real-time path analysis and retention rate calculation, and optimize marketing strategies. High-concurrency queries during major promotions: Use real-time local cache to reduce remote storage access and meet real-time analysis requirements under instantaneous high load. Temporary special analysis: Use a one-time strategy to load cache for a specified date range (such as activity data from the past 7 days or quarterly cycle data), avoid repeated remote storage reads, improve short-term high-frequency review efficiency, and adapt to temporary and non-periodic analysis needs.
Gaming	Process game logs and player behavior data in real time to support operational decision-making and optimize user experience.	Real-time game log analysis: Cache high-frequency query data such as player online duration and item usage to enable the operations team to quickly respond to abnormal situations (such as cheat detection). Live interaction analysis: Preload live room user interaction data (such as bullet comments and gift rewards) to accelerate real-time popularity leaderboard generation. Advertising effectiveness evaluation: Cache historical advertising data to quickly analyze the ROI of different channels and optimize advertising budget allocation. Version update effectiveness analysis: Accurately load game data for specific time frames (such as version update windows) to improve the efficiency of effectiveness evaluation.
Automobile	Vehicle network devices (such as sensors, GPS, cameras, and more) generate vehicle status, driving behavior, and environmental information in real time. Quickly process and analyze these data to support real-time monitoring, fault diagnosis, and user services.	Historical data acceleration: Use offline cache strategies to preload frequently accessed historical data (such as driving behavior data from the past 30 days), accelerating the generation of driving reports or user behavior analysis. Query acceleration: When generating driving scores, vehicle health reports, or user behavior analysis, directly read data from local cache to reduce the latency of remote storage access. New function verification analysis: After the new function is launched, vehicle data from the first two months of use by the initial batch of car owners is centrally analyzed through one-time cache loading to verify user acceptance and actual performance of the new car function, assisting car companies in quickly iterating and optimizing the function.

Description

The ByteHouse cache loading task preloads data stored in the database into each virtual warehouse, as shown in the figure below.

Cache loading tasks are available in three modes: real-time loading task, periodic loading task, and one-time loading task. The comparison is as follows.

Comparison items	Real-time loading task	Periodic loading task	One-time loading task
Loading task description	The latest written data is preloaded in real time to the virtual warehouse cache. The same table only supports real-time loading to one virtual warehouse.	Data is loaded into the virtual warehouse cache periodically. Supports loading one table to multiple virtual warehouses.	A one-time task that loads data within the specified date range to the virtual warehouse cache. Supports loading one table to multiple virtual warehouses.
Data loading logic	When data is written into the table, it triggers the cache loading task, and the newly written data is loaded into the cache.	Runs at a fixed time every day: Partition table: Date-type partition table: Refreshes the incremental data of partitions within the TTL period each time it runs. Non-date-type partition table: Refreshes the entire table each time it runs. Non-partition table: Refreshes the entire table each time it runs.	Loads only on the specified date: Partition table: Date-type partition table: Loads data within the specified date range into the cache. Non-date-type partition table: Loads the entire table data each time it runs. Non-partition table: Refreshes the entire table each time it runs.

Comparison items

Real-time loading task

Periodic loading task

One-time loading task

Loading task description

The latest written data is preloaded in real time to the virtual warehouse cache. The same table only supports real-time loading to one virtual warehouse.

Data is loaded into the virtual warehouse cache periodically. Supports loading one table to multiple virtual warehouses.

A one-time task that loads data within the specified date range to the virtual warehouse cache. Supports loading one table to multiple virtual warehouses.

Data loading logic

When data is written into the table, it triggers the cache loading task, and the newly written data is loaded into the cache.

Runs at a fixed time every day:

Partition table:
- Date-type partition table: Refreshes the incremental data of partitions within the TTL period each time it runs.
- Non-date-type partition table: Refreshes the entire table each time it runs.
Non-partition table: Refreshes the entire table each time it runs.

Loads only on the specified date:

Partition table:
- Date-type partition table: Loads data within the specified date range into the cache.
- Non-date-type partition table: Loads the entire table data each time it runs.
Non-partition table: Refreshes the entire table each time it runs.

Limitations

The engine version must be 2.3 or above to use the cache loading function. The materialized view cache loading function requires engine version 2.3.2 or above. You can view the current engine version on the Tenant management > Basic information page.
Materialized view cache loading is a Beta function. To use it, submit a ticket or contact the ByteHouse team to obtain allowlist privileges.

Important notes

The cache loading task will cache table data into the cache resources of the virtual warehouse. The virtual warehouse is configured by default with a certain amount of cache space. You can also enable the local disk feature of the virtual warehouse (value-added feature) to obtain a larger cache resource space. For details, please refer to Use the virtual warehouse local disk.

Create cache loading task

After logging in and entering the ByteHouse cloud data warehouse console, click the top Tenant management, then enter the Preload page, and click the upper right Create Preload Task.

Configure the cache loading task in the pop-up page.

Configure basic information. Configure the task name and description in the interface.

Loading configuration. Select the task type for cache loading and configure task details.

Param	Configuration description
Task type	Currently supports real-time loading, periodic loading and one-time loading as three task types. Real-time loading: The latest written data is preloaded in real time to the virtual warehouse cache. One table only supports real-time loading to one virtual warehouse. Periodic loading: Data is loaded into the virtual warehouse cache according to the set period. For tables with partitions not at the day level, the entire table data will be loaded. Supports loading one table to multiple virtual warehouses. Period load Mission also requires further configuration of the data scope, load time, and other mission parameters below. Single load: Data is loaded into the virtual warehouse cache in one batch according to the date range. For tables without day-level partitioning, the entire table data will be loaded. Supports loading a single table into multiple virtual warehouses. Single load Mission also requires further configuration of the date range parameters below.
Data scope	Configure the period load mission to load data from the most recent number of days.
Load time	Configure the scheduled execution time for the period load mission each day.
Timezone	Configure the timezone used for the period load mission.
Immediately load	Select whether to launch a load mission immediately after the period mission is created.
Date range	Configure the date range for the single load mission.

Configure loading objects. In the dropdown box, select the database, data table, materialized view to preload, and the virtual warehouse to write to.
tip
Real-time cache mission only supports selecting one virtual warehouse, that is, only supports preloading a table into one virtual warehouse.

After configuration is complete, click Create.

View cache loading task

For one-time loading and periodic loading tasks, after the cache loading task is created, you can click View Executions on the cache loading task list page to view the specific task's running status.

On the mission status details page, you can view whether the loading mission was successful. You can also view the loading duration, partition-level status display, and loading status for each table or partition. For missions that are partially successful, ByteHouse displays the percentage of success. You can expand the table to view the reasons for loading failures. ByteHouse also supports retrying loading missions. You can retry missions for each day, with options to retry the entire table or select a single partition of the table for retry.

Edit cache loading missions

You can also edit and modify created cache loading missions on the cache loading mission list page.

Cache strategy	Edit change important notes
Real-time cache strategy	When adding or unchecking tables to be loaded into the cache: After unchecking a table, data from that table will no longer be cached in the virtual warehouse. Previously cached data will be gradually deleted according to the cache data elimination strategy (LRU strategy). For newly checked tables, subsequent data will be cached in the virtual warehouse according to the current cache strategy. When switching the virtual warehouse for cached data: After switching the virtual warehouse, subsequent data will be cached in the new virtual warehouse. Previously cached data in the original virtual warehouse will be gradually deleted according to the cache data elimination strategy (LRU strategy).
Offline cache strategy	When adding or unchecking tables to be loaded into the cache: After unchecking a table, data from that table will no longer be cached in the virtual warehouse. Previously cached data will be gradually deleted according to the cache data elimination strategy (LRU strategy). For newly selected tables, the current caching strategy will be followed, and the table's historical data will be periodically loaded to the virtual warehouse. When adding or reducing the number of virtual warehouses for cached data: After adding a new virtual warehouse, subsequent data will be cached in the new virtual warehouse. After reducing the number of virtual warehouses, no new data will be cached in the virtual warehouse. Historical cached data will be gradually deleted according to the cached data elimination strategy (LRU strategy).
Single-use caching strategy	When adding or deselecting tables to be loaded into the cache: After deselecting a table, subsequent data from that table will no longer be cached in the virtual warehouse. Previously cached data will be gradually deleted according to the cached data elimination strategy (LRU strategy) For newly selected tables, the current caching strategy will be followed, and the table's data will be cached in the virtual warehouse. When switching the virtual warehouse for cached data: After switching the virtual warehouse, subsequent data will be cached in the new virtual warehouse. Cached data in the previous virtual warehouse will be gradually deleted according to the cached data elimination strategy (LRU strategy).

Delete cache loading missions

You can also delete created cache loading missions on the cache loading mission list page.

After deleting a cache mission, the cached data is not deleted immediately. It will be gradually cleaned up later according to the elimination strategy (LRU strategy).

Manually delete cached data

ByteHouse supports manually cleaning cached data for tables or partitions as needed.

warning

Before emptying the cache, confirm whether the cached data has been used to avoid accidental deletion.

Click View Executions, then click Empty cache.

View cache monitoring

After you create a cache mission, you can view metric data such as the cache utilization rate and the number of cached data read and write operations for the local disk of the virtual warehouse on the Tenant management > Cloud monitor page, and perform routine monitoring of the local disk of the virtual warehouse.

For details on the currently supported monitoring metrics for the local disk of the virtual warehouse, as well as guidance on creating alarms through BytePlus cloud monitoring, see Alarm.

Last updated: 2026.03.31 16:23:11