The query of agent will be based on the data of the dataset configured in this step, and the user can customize the scope of the dataset queried by the agent, so that the agent can be better applied to the current business. This article provides you with configuration guidance related to creating datasets that agents can query when configuring an Intelligent Analysis Agent.

The current agent's data source configuration supports two data sources:

• Add the created dataset as a data source for the agent.
• Add a visualization chart created in Datawind (DataWind) as a data source for the agent. This approach has been supported since version 3.2.0, and you need to combine the DataWind products.

This article describes the activity details for configuring an agent data source based on a created dataset; see DataWind for creating a data source based on a chart Chart Configuration.

Before configuring the data source of the agent, you need to create the corresponding data as a dataset of the DataAgent to complete the data preparation operation. For details, please refer to Data Preparation Overview.

After entering the agent management configuration page, click "Data and Knowledge" in the left navigation bar. After entering the data source configuration page, you can start the configuration of the dataset in the "Dataset" tab.
As of version 3.17, adding a dataset and initializing will enable dataset vectorization by default, see below for details Dataset initialization.

Manually select dataset #

You can click "Add Dataset" on the page to add the dataset you have permission for.

AI suggestions on configuration #

The Intelligent Analysis Agent provides you with the function of "Optimizing Dataset Configuration". You can upload some questions and the AI will give you suggestions for dataset configuration. When using this function,

1. Click the "Optimize Dataset Configuration" button, download the problem template according to the interface prompts, fill in the "Dataset Name" and possible "Questions", and then click "Batch Upload Questions". The product will help users optimize the dataset configuration according to the possible questions users have about the dataset.
   
2. After the upload is successful, click "OK", wait for the optimization suggestion to be generated, and click "View".
   

Configure semantic model #

Configure key points #

Operation guidance #

1. Click Config Model on the Dataset Configuration page.
   

2. Click "Set basic model information" to add a name, description, and features to the semantic model.
   
   The relevant parameter configuration instructions are as follows:

   Parameter	Configuration instructions
   Semantic model name	Support user-defined semantic model names. If not specified, they will be synchronized with the dataset name by default.
   Semantic model description	Support users to add descriptions to the semantic model. If not specified, it will be synchronized with the dataset description by default.
   Feature	The feature values of the semantic model are used to define the key characteristics that distinguish the current dataset from other datasets. Users can use commas and carriage returns to separate multiple values. warning Features affect the priority of the agent surface recall dataset. A single feature is recommended to be no more than 5 characters in length, and the total number of features is recommended to be no more than 10. Feature examples: aaa, bbb, ccc.

3. After the field list of the current dataset, click the "Configure" button of a field to enter the semantic model configuration interface of the corresponding field.
   
   Fields of different data types involve some differences in the configuration parameters, which are described below.

   • General configuration parameters

     Parameter	Configuration instructions
     Agent field name	The user can customize the agent field name. When the description at the time of query is this name, the agent will preferentially use this field for query. When empty, it is consistent with the field name of the dataset.
     Field name synonym	The user can set one or more synonyms for the field name, and when the description of the query is in these synonyms, the agent will preferentially use the field for query. Multiple synonyms are separated by commas or carriage returns.
     Field description	The user can add a description of the agent field to help understand the field, which is consistent with the field description of the dataset when empty.
     Default recall	If "Default Recall" is checked, this field will be passed to the large model as a prompt word each time a question is asked.

   • Text class field-specific configuration parameters

     Parameter	Configuration instructions
     Field value recall	Text class fields turn on the field value recall button by default, you can close it on demand, and the field value will not be recalled after closing.
     Field value blacklist	Field values that do not want to be recalled
     High frequency field value description	For high-frequency text fields, the field meaning range can be configured. The configuration example is as follows.

   • Numerical class field-specific configuration parameters

     Parameter	Configuration instructions
     Default calculation method	In the drop-down box, select the default calculation method for field values of the numeric class. Current support: sum, count, count differently, mean, maximum, minimum.

4. Enables fields that require the use of a semantic model.
   You can search to open the semantic model configuration content of the required fields in the dataset one by one, or click "Batch Open Status" at the bottom of the field list to open the semantic model configuration of all fields in the dataset.
   

Configuration Knowledge Base #

Configuration points #

Operation guidance #

1. Click on the Associated Knowledge" of the dataset configuration to enter the knowledge management configuration page.
   

2. Click "Knowledge Base Configuration" to configure parameters such as matching degree preset and recall upper limit of the knowledge base.
   
   The relevant parameter configuration instructions are as follows:

   Parameter	Configuration instructions
   Match threshold	Adjust the vector matching degree between business terms and problems, with a minimum of 0 and a maximum of 1, indicating an accurate match. (Users who are not familiar with large models are advised not to adjust.)
   Upper limit of knowledge recall	Control the maximum number of one-time recall knowledge in the current dataset to avoid the interference of too much recall knowledge on the large model. (It can be configured, but not too much is recommended. The range of 2-5 is better)

3. Click "Add Knowledge" to add more knowledge to the knowledge base.
   
   
   The knowledge of a dataset can usually be used to describe a data range after combining data set fields. The relevant parameter configuration instructions and examples are as follows:

   Parameter	Configuration instructions	Configuration example
   Business term	Business terms such as specific nouns, slang, and names of common terms, when the recall rule is "follow the knowledge base configuration", the business term will be used as a keyword to recall the piece of knowledge.	For example, for a crowd profile dataset, when you need to disassemble the crowd field by group profile: Urban white-collar population: City field (Beijing, Shanghai and Guangzhou) + Age field (20~ 50) + Income field (2~ 6w)
   Conjunction	It is used to connect business terms and descriptions, including "representation" and "none" connectors. The function of connectors is mainly to make the whole sentence smoother, and whether connectors are needed can be determined according to the smoothness of previewing the whole sentence knowledge.
   Explain	The definition, interpretation, etc. of input knowledge are the core parts of knowledge, and the description of knowledge needs to be accurate and easy to understand to help the large model understand more accurately. For example, the description of the business term "search performance caliber" is "Definition: orders/traffic are counted in searches".
   Synonym	Synonyms of business terms serve the same purpose as business terms. For example, the synonyms of age distribution include age profile, age composition, and age structure.
   Preview	You can check whether the overall knowledge is smooth through the preview. This is the knowledge that is ultimately given to the model, including the concatenation of business terms, connectors, and descriptions.
   Recall rule	Includes following knowledge base configuration, default recall, and custom rules. Following the knowledge base configuration means matching knowledge based on the similarity of business terms or synonyms Default recall means that this piece of knowledge is transmitted to the large model as a prompt word every time a question is asked The custom rule indicates that according to the keyword matching rule in the user's question, it can be set to include keywords and not include keywords. Keywords can be filled in multiple, and the relationship between multiple keywords is "and".
   Participate in the recall of field names	Field name recall refers to the current knowledge, whether to participate in field recall, the way to participate is to recall according to the field name in the knowledge. For example, the name of knowledge: sales represents sales and profit margins. Then the knowledge participates in the recall, and the two fields of sales and profit margin will be recalled.
   Participate in recall field value	Field value recall refers to the current knowledge, whether to participate in the field recall, the way to participate is to recall according to the field value in the knowledge. For example, the name of knowledge: Douyin, watermelon, Jinri Toutiao. If this knowledge participates in the recall, the field "Product Name" will be recalled according to the values of Douyin, Watermelon, Jinri Toutiao, etc.

Configure filter #

1. The user clicks "Filter Configuration" to enter the dataset filtering configuration interface, and can filter the dataset data used by the agent according to the dimension field or indicator field in the dataset.
   

2. Set the filter criteria for the dataset used by the agent, and users can use dimension filtering, metric filtering, or date filtering based on the desired data situation.
   

   Filter criteria	Applicable scenario	Operation guidance
   Dimension filter	The data needs to be filtered by dimension field to distinguish between dimension and metric field pairs, seeDimensions and Metrics.	Dimension Filtering
   Metric filter	The data needs to be filtered according to the indicator fields, and the distinction between dimensions and indicator fields is referred to in Dimensions and Metrics.	Metric Filtering
   Date filter	Data needs to be filtered by date type fields	Date Filtering

Configure dataset readiness #

Click "Config Readiness" to enter the configuration dataset ready interface. You can set the synchronization date and time for dataset readiness. After setting, click "Save".

• When the Lark-side active push triggers, it will detect whether the time partition data of the T-N is ready, and the partition data is ready to start pushing.
• Example: Set to T-2, on the 2020-10-05 day, complete the "2020-10-03" day data synchronization, then the relevant active push starts to push.

Import semantic models/knowledge bases from other agents #

Starting with the 3.5.0 release, direct import of semantic models and knowledge base configurations for other agents is supported. When you add a dataset to an agent and configure the semantic model and knowledge base of the dataset, if you need to use the same dataset when creating a new agent later, and the field semantic model and knowledge base of the dataset are similar to the previous configuration, you can directly import the corresponding configuration into the newly created agent.

1. The newly created agent has added datasets that other agents have added and configured.
2. On the new agent's dataset page, you can click the "Import from other agents" button at the top right of the page.
   
3. In the pop-up window, select the agent and dataset where the semantic model and knowledge base to be imported are located, and tick the semantic model or knowledge base to be imported.
   • Import the semantic model:
     • The semantic model configuration for each field in the source agent dataset (agent field names, synonyms, recall switches, etc.) is synchronized to the current agent dataset.
     • The model base information (semantic model names, features, etc.) set by the source agent dataset is not synchronized to the current agent, so after importing, you need to manually set the model base information of the current agent dataset on demand.
   • Import Knowledge Base:
     • The pieces of knowledge (business names, connectors, descriptions, etc.) added to the source agent dataset are synchronized to the current agent's dataset.
     • The knowledge base configuration (match threshold, knowledge recall upper limit) set by the source agent data is not synchronized to the current agent, so after importing, you need to manually set the knowledge base configuration of the current agent dataset on demand.
4. Check Finish and click "OK" to complete the import.

After completing the dataset configuration, you can view the data set initialization status in the list and view the enabled status of each data set in scenarios such as smart quizzes, in-depth research, etc.

Dataset initialization #

After adding a dataset, the system automatically initializes the dataset and performs data set vectorization automatically. During the initialization process or after the initialization is completed, you can click the status column "Initializing" or "Completed" status, click "View Details", and jump to the initialization configuration page to view the configuration details.

The example of the initialization configuration interface is as follows:

Dataset initialization also has two states: incomplete and failed. You can try the initialization configuration again according to the interface prompts:

• Not completed: Initialization has not yet started, you need to click "Not completed" status, click "Go to initialization configuration", and complete the data set initialization settings according to the prompts.
• Failed: The initialization of the current dataset failed. You can click the "Failed" status, and contact the user with edit permissions above to complete the initialization configuration according to the prompts in the pop-up window.

Turn dataset vectorization on/off #

After the initialization of the dataset is completed, the vectorization of the dataset is turned on by default. If the subsequent dataset does not need to be used by the current agent query, you can click "Completed" in the status column, jump to the initialization configuration page, and turn off the vectorization.
After closing, the configuration of the dataset is preserved, but subsequent use of the agent will no longer query the data in the corresponding dataset.

Enable/Disable Dataset Application Scenarios #

After the dataset is created, Intelligent Data Query and In-depth Research are turned on by default, and you can turn on/off the corresponding switches on demand in the list.

• Support for deleting datasets that have been added.
  
• If the current dataset has already been used in a recommendation question, you need to modify the recommendation question before deleting the current dataset.