ByteHouse supports creating external catalogs, including Hive, Iceberg, Paimon, and MySQL. It supports displaying and managing external data catalogs, adding and deleting catalogs, viewing database and table information within catalogs, and configuring catalog resources and data permissions. This article introduces how to create and manage external Catalogs, and grant users relevant data and resource permissions.

ByteHouse External Catalog is used to manage the information and resources of external data sources you created. The system currently supports creating and managing four types of data sources: Hive, Iceberg, Paimon, and MySQL. This is suitable for scenarios that require unified query and management of data stored in various external data sources, such as Hive, Iceberg, Paimon, and MySQL, without the need for data migration. Description of metadata storage types supported by different data sources as follows:

Through the external Catalog, you can directly access and manage these external data sources. The specific feature details are as follows:

After adding a new external Catalog, all ByteHouse internal Catalogs are migrated to the default folder.

Before using an external Catalog, obtain the following information according to the metadata storage type used by your Catalog. To add MySQL Catalog, only MySQL-related information is required; there is no need to pay attention to other information in the table below.

This feature is in Beta. Please submit a ticket or contact the ByteHouse team to obtain allowlist privileges.

1. Log in to the ByteHouse Cloud Data Warehouse Console. On the top Database tab, click + New. Then, click Create external catalog.
   

2. Select the external Catalog type and the metadata storage type, and configure the external Catalog information.

   • For Hive, Iceberg, and Paimon catalog types, you can view the corresponding configuration information based on the metadata storage type.
   • MySQL Catalog does not require configuration of a metadata storage service. You can view the connection information that needs to be configured.
     
     Parameter configuration information when the metadata storage type is Hive, Hadoop, or Filesystem:
     For Catalogs with metadata storage types Hive, Hadoop, or Filesystem, the parameter types required for configuration are the same, but the methods for obtaining them differ. For details, see the following parameter configuration description.

   Param	Hive	Hadoop	Filesystem
   Metadata storage service address / Storage service address	The connection address of the Hive Metastore service. Hive Metastore is a service that stores Hive table metadata, such as table structure and partition information. The format for filling in and examples are as follows: Format: thrift://hive_thrift_server_ip:port, where <hive_thrift_server_ip> is the IP address of the Hive Thrift server and is the port number. You can refer to the official documentation for your Hive service to obtain connection details. Example:thrift:// 101.1 xx.xx. xxx:9 083	Enter the connection address of the storage service used by Hadoop. You must provide the actual storage path, and you can use S3 or HDFS service addresses. Format for filling in and sample are as follows Format: S3: s3://<bucket_name>/<path_to_metadata>. For BytePlus TOS, use the tos://<bucket_name>/<path_to_metadata> format. <bucket_name> is the name of the bucket; <path_to_metadata> is the path of metadata in the bucket. HDFS: hdfs://<namenode_host>:<port>/<path_to_metadata>, <namenode_host> is the hostname or IP address of the namenode in the Hadoop cluster; <port> is the port number. <path_to_metadata> is the specific path in HDFS for storing metadata. Example: s3://bytehouse-test/iceberg tos://test-catalog/iceberg hdfs://haruna/home/bytehouse_test/table_store/bytehouse_test	Enter the actual storage path as the connection address for the storage service used by the filesystem. S3 or HDFS service addresses are supported. Format for filling in and examples Format: S3: s3://<bucket_name>/<path_to_metadata>. If you are using BytePlus TOS, you can use the tos://<bucket_name>/<path_to_metadata> format. <bucket_name> is the bucket name; <path_to_metadata> is the path to the metadata in the bucket. HDFS: hdfs://<namenode_host>:<port>/<path_to_metadata>, where <namenode_host> is the hostname or IP address of the Hadoop cluster's namenode. <port> is the port number; <path_to_metadata> is the specific path in HDFS for storing metadata. Example: s3://test-catalog/paimon tos://test-catalog/paimon hdfs://haruna/home/bytehouse_test/table_store/bytehouse_test
   Service endpoint	Configure as the endpoint for the S3-compatible storage service. If you are using BytePlus TOS, refer to the list of available service endpoints in the S3 Endpoint column of the Region and URL Full Domain table.
   Access Key (AK)	The Access Key ID of the S3-compatible storage service. For information on how to obtain it, refer to the key management documentation of the corresponding service. If you are using BytePlus TOS, refer to AccessKey (Key) Management for instructions on how to obtain an AccessKey.
   Secret access key (SK)	Secret Access Key of the S3-compatible storage service. For instructions on how to obtain it, refer to the key management documentation of the corresponding service. To obtain an AccessKey when using BytePlus TOS, refer to AccessKey Management.
   Catalog name	Customize the Catalog name: naming rules are as follows A name can only contain letters, numbers, and the underscore "_". Length must be 1 to 64 characters.

   Parameter configuration information when the metadata storage type is LAS:

   Param	Configuration description
   Service endpoint	Define the TOS Service endpoint used by LAS. For a list of available service endpoints, refer to the information in the S3 Endpoint column in Region and URL Full Domain (Endpoint).
   TOS access key (AK)	Configure as the access key for TOS. For information on how to obtain an AccessKey, please refer to AccessKey (Key) Management.
   TOS access key (SK)	Configure as the secret key for TOS. For information on how to obtain an AccessKey, see AccessKey (key) management.
   LAS access key (AK)	LAS access key. To learn how to obtain an AccessKey, see AccessKey (Key) Management.
   LAS secret access key (SK)	Secret key configured for LAS. For information on how to obtain an AccessKey, see AccessKey (Key) Management.
   LAS Catalog name	Please enter here the name of the Catalog that you wish to connect to, which already exists in LAS. This allows ByteHouse to locate the correct external data directory. Log in to the LAS Console, and in the left navigation pane, click Database > Metadata Management to view the Catalog name.
   LAS region	The region where the physical data center of the TOS storage service used by LAS is located. For available region names, see Region and URL Full Domain.
   LAS metadata storage service URL	Configure the LAS metadata storage service address. You can fill in thrift://lakeformation.las.{REGION}. ivolces.com:48869 or thrift://lf_thrift_server_ip:port. For information on how to obtain the address, see配置外表Descriptions of connection addresses in LAS
   Catalog name	Naming rules for custom catalog names: The name can only contain letters, numbers, and the underscore "_". The length is 1 to 64 characters.

   MySQL Catalog configuration information:

   Param	Configuration description
   Connection address	The connection address of the MySQL service. The format is IP:port. ip is the IP address of the MySQL server, and you can refer to the official documentation for your MySQL service to obtain the IP address. port is the port number of the MySQL service. You can refer to the official documentation of the MySQL service you use to obtain the port number.
   Username	You can refer to the official documentation of the MySQL service you use to obtain the username.
   Password	You can obtain the password for your MySQL service by referring to its official documentation.
   Catalog name	Customize the Catalog name. The naming rules are as follows: The name can only contain letters, numbers, and the underscore "_". Length is 1 to 64 characters.

3. Click Create.

In the left navigation bar under the Database tab, you can view the newly created Catalog. Through the External Catalog, you can view databases, tables, DDL, data preview, and other information.

• View database
  
• View data table
  
• Table data preview
  

In the left navigation bar, click the name of the external catalog folder you want to delete to open its details page. Click .... Click Delete Catalog. Click Apply. The Catalog will then be deleted.

You can manage permissions for the External Catalog by granting roles to users or by directly granting data and resource permissions. For activity details, please refer to the following documents:

• User Management
• Role management
• Data access control
• Resource Access Management