ByteHouse Cloud Data Warehouse provides SQL audit capabilities and an interface for viewing SQL execution logs. It supports viewing SQL history and running queries, helping you quickly locate and optimize slow queries, troubleshoot errors, and audit user behavior to ensure data security. These features are suitable for database performance tuning, security compliance audits, post-incident tracing, and other scenarios. Additionally, ByteHouse provides a detailed SQL log table. After enabling the detailed log toggle, you can view the corresponding detailed SQL log table data to further inspect more granular log data. This article introduces how to use the ByteHouse SQL audit feature.
The ByteHouse SQL audit feature supports viewing SQL history and running queries. In SQL history, you can view slow SQL queries, error queries, and perform precise searches by ID or SQL statement content.
This feature provides comprehensive audit dimensions, including query ID, status, username/ID, effective roles, virtual warehouse, start/end time, duration, SQL type, SQL statement, total CPU time, maximum memory usage (Segment), and others. Additionally, this feature supports a wide range of filter options, making it convenient for you to efficiently analyze SQL execution status.
In addition, ByteHouse also supports viewing detailed logs. After enabling this feature, ByteHouse will begin accumulating data (a new connection to ByteHouse is required) and will automatically create a database named bh_system and a query_log table within it to record detailed SQL execution log data. You have all privileges on this database and table. The default data lifecycle of this table is 7 days. To retain data for a longer period, you can execute SQL commands to delete the TTL as needed or modify the TTL time. For operation details, see Adjust the query_log table lifecycle.
tip
You can also use the SQL command SHOW PROCESSLIST to view currently running queries. For usage instructions, see SHOW PROCESSLIST.
warning
After obtaining Allowlist privilege, you can open the Enable detailed logging toggle at the top right of the Query history page. Once enabled, ByteHouse will create a query_log table in the system for writing detailed SQL execution log data.
bh_system database under your [tenant-environment], and create a query_log table in this database. You have full privileges for this database and table. To allow sub-users to view the query_log, contact the administrator to grant data privileges. For operation details, refer to Data privilege management.warning
Because ByteHouse continuously writes the SQL log information you run into the bh_system.query_log table, any actions such as adding, deleting, or modifying this table will affect log writing. Except for modifying the TTL, adding, deleting, or modifying this table is prohibited.
query_log table. You can view the query_log table by clicking Database at the top of the page or Data Query.query_log table using the SQL worksheet, you can see this table in the bh_system database under the data objects on the left side of the SQL worksheet.The bh_system.query_log table has a default data lifecycle of only 7 days. To retain data for a longer period, refer to the following commands to delete the TTL or modify the TTL time as needed.
# 查看 bh_system.query_log 表当前生命周期 SHOW CREATE TABLE bh_system.query_log; # 删除 TTL ALTER TABLE bh_system.query_log REMOVE TTL; # 将默认 TTL 从 7 天改为 180 天(您可以将 180 替换为您希望保留日志的天数,需管理员权限执行) ALTER TABLE bh_system.query_log MODIFY TTL event_time + INTERVAL 180 DAY;
Column name | Data type | Field description |
|---|---|---|
type | Enum8, enumeration values include:
| Query event type:
|
event_date | Date | Date the event occurred |
event_time | DateTime | Precise event occurrence timestamp |
event_time_microseconds | DateTime64(6) | Event timestamp (microsecond precision) |
query_start_time | DateTime | Query start time |
query_start_time_microseconds | DateTime64(6) | Query start time (microsecond precision) |
query_duration_ms | UInt64 | Total query duration (ms) |
read_rows | UInt64 | Total number of rows read from all tables and table functions in the query |
read_bytes | UInt64 | Total number of bytes read from all tables and table functions in the query |
written_rows | UInt64 | Number of rows written (only INSERT actions) |
written_bytes | UInt64 | Number of bytes written (only INSERT actions) |
result_rows | UInt64 | Number of rows returned in the result |
result_bytes | UInt64 | Number of bytes returned in the result |
memory_usage | UInt64 | Highest memory usage by the query on each worker and each segment |
current_database | String | Current database |
query | String | Complete query statement |
normalized_query_hash | UInt64 | Normalized query hash value |
query_kind | String | Query type (such as |
databases | Array(String) | Database(s) involved in the query |
tables | Array(String) | Table(s) involved in the query |
columns | Array(String) | Columns of the table(s) involved in the query |
partition_ids | Array(String) | Partition(s) |
projections | Array(String) | List of projection names involved in the query |
materialized_views | Array(String) | List of materialized view names involved in the query |
exception_code | Int32 | Exception code (only populated when |
exception | String | Exception information (error message) |
stack_trace | String | Exception stack trace information (requires configuration |
is_initial_query | UInt8 | Indicates whether this is the initial query (0: subquery; 1: query initiated directly by the user) |
user | String | Username |
query_id | String | Query ID |
interface | UInt8 | Interface used to initiate the query. Possible values:
|
os_user | String | Operating system user running the TCP client |
client_hostname | String | TCP client hostname |
client_name | String | TCP client name |
client_revision | UInt32 | TCP client revision |
client_version_major | UInt32 | TCP client major version |
client_version_minor | UInt32 | TCP client minor version number |
client_version_patch | UInt32 | TCP client patch version |
http_method | UInt8 | HTTP method used to initiate the query. Possible values: |
http_user_agent | String | User-Agent information of the client |
http_referer | String | HTTP referer information |
forwarded_for | String | Client IP in the HTTP proxy chain |
quota_key | String | Quota key |
revision | UInt32 | ByteHouse revision |
ProfileEvents | Map(String, UInt64) | PrunedPartitions: Number of partitions actually read. warning The TotalCPUTimeMs column is only supported in ByteHouse CDW engine v2.3.x and above. Please confirm the engine version before use. You can log in to the ByteHouse Console, click the top Tenant Management, and view the engine version you are using on the Basic info page. |
Settings | Map(String, String) | Effective configuration parameters for the query (those overriding default values) |
log_comment | String | Log comment |
thread_ids | Array(UInt64) | List of thread IDs participating in the query |
used_aggregate_functions | Array(String) | Aggregate functions used |
used_aggregate_function_combinators | Array(String) | Aggregate function combinators used |
used_database_engines | Array(String) | Table engines used |
used_data_type_families | Array(String) | Data type families used |
used_dictionaries | Array(String) | Dictionaries used |
used_formats | Array(String) | Data format parameters used (for example: FORMAT CSV) |
used_functions | Array(String) | Functions used |
used_storages | Array(String) | Storage systems used |
used_table_functions | Array(String) | Table functions used |
virtual_warehouse | String | Computing group used for the query |
query_plan | String | Query execution plan |