Background: When using the "Add Field" feature, users will notice that there are two types of fields in the list on the left. One is a data source field, and the other is a dataset field. So, what is the difference between these two types? Which type of field should users select when writing expressions?

1.1 What is a data source field? #

A data source field, as the name suggests, is a field that comes from the data source. For example, in the dataset shown below, the data source is Feishu Sheets, and the fields contained in Feishu Sheets itself are data source fields.

1.2 What is a dataset field? #

A dataset field is a field contained in the dataset. These fields are not fields from the data source itself, but rather fields automatically created by the dataset, as well as fields newly created by users using expressions.

1.3 What is the difference between data source fields and dataset fields? #

Data source fields cannot be queried directly. On the left side of the visual query page, all fields displayed in the field list are dataset fields.

After the data source of a dataset is parsed, the dataset will automatically generate a dataset field for each data source field on the field configuration page. The expression for each of these initial dataset fields is the corresponding data source field itself.
For example, in the figure below, the expression for the "Payment Type" dataset field is the 'Payment Type' data source field.

1.4 What is the difference between data source fields and dataset fields with the same name? #

The result represented by a data source field is the original result from the data source, while a dataset field must follow the specific expression of the field.
The following example helps illustrate this:
When the expression of a dataset field is the same as that of a data source field, there is no difference between the dataset field and the data source field. For example, in the figure below, the "order_date" dataset field has an expression that is simply the order_date field from the data source, so the results of the two are the same.

However, since the expression of a dataset field can be modified arbitrarily, some dataset field expressions may be changed later, resulting in their expressions being different from the original corresponding data source fields.
For example, in the figure below, the "product_type" field may have originally had the expression product_type, but it has now been changed to count(product_type). At this point, the meanings represented by these two fields are completely different. To use the original meaning of "product_type", select the data source field.

As for user-created fields, since there are no corresponding data source fields, this issue does not arise.

Background: There are two main types of fields in a dataset: dataset fields and personal fields. Dataset fields can also be referred to as public fields.

2.1 What are public fields and what are personal fields? #

Public fields are fields in the dataset that are directly added to the dataset. Every user with the appropriate permissions can view and use these fields. For example, the fields visible to all users in the figure below are public fields.

Public fields are fields that belong to the dataset itself. Adding a public field is equivalent to modifying the dataset, so only users with dataset editing permissions can add public fields.
Because the content of a dataset cannot be easily modified, dataset editing permission is a relatively high-level permission that only a few users have.
However, when users are working with this dataset and want to add fields, they can add personal fields. Personal fields do not affect the functionality of the dataset itself.

2.2 How to choose whether to add a public field or a personal field? #

When adding a field, there is an option labeled "Save to dataset" in the lower left corner. If this option is selected, the field will be saved to the dataset and become a dataset field, that is, a public field.
If it is not selected, a personal field will be created.

2.3 What are the characteristics of personal fields and how can they be distinguished? #

(1) Carefully observe the diagram below: "order_date" is a public field, while "province_sale" is a personal field. When personal fields appear in the field list or are placed in the field bar, the font is italicized. Additionally, in the field bar, the font color is blue.

(2) When adding fields, both public and personal fields can be referenced. However, when a personal field is referenced, it is prefixed with [Personal] to indicate that it is a personal field. For example, as shown in the figure below, the first row is a public field, and the second row is a personal field.

(3) Personal fields are only displayed in the creator's field list and cannot be found by other users through search.
However, DataWind supports sharing query links with other users. If a personal field is used in the current user's query and then shared with other users, what impact does this have?
As shown in the figure below, other users can view the current user's personal fields in the chart. However, in the field bar, the font color of personal fields is gray, and the font is italicized. Other users do not have permission to search for the current user's personal fields, but they can perform operations such as dragging the corresponding field from Dimensions to Metrics.

2.4 Why does directly copying another user's expression to create a field result in an error? #

Personal fields can only be referenced by their creator. If an expression references another user's personal field, directly creating a field using this expression will result in an error.

2.5 How to view the expression of another user's personal field? #

Users with dataset editing permissions can view only those personal fields of other users that have been placed in a chart. The user can click the downward triangle in front of the field bar, select "Edit field" from the expanded options, and then view the expression of this field.

If the current personal field is in the filter bar and there is no "Edit field" option, the user can drag the current field to the dimension or metric bar, then select "Edit field" to view it.

2.6 Summary #

3.1 What is a field expression? #

A field expression mainly consists of fields and functions, and may also include constants or parameters. To reference other fields, the referenced field must be searchable in the field list on the left. Common functions are also provided on the right.

3.2 What is the difference between dimensions and metrics? #

There is no clear boundary between dimensions and metrics. For example, in a class grade dataset, to view each student's grade in the class, the student's name is the dimension and the grade is the metric. To view the number of students in each score segment, the score is the dimension and the number of students is the metric.

3.3 Why can't some fields be dragged to the dimension? #

Check whether the current field is an aggregate field. Aggregate fields cannot be used as dimensions.

3.4 What is an aggregate field? #

Simply put, if a field's expression contains an aggregate function or references another aggregate field, then this field is an aggregate field.

3.5 What is an aggregate function? #

Aggregation refers to combining multiple data items into a whole through certain calculation methods. For example, the most common sum function is used to calculate the sum of a set of data to obtain a single value, which is essentially an aggregation process. Other functions such as count, avg, max, min, quantile, and groupArray are also aggregate functions. When any of these functions are used in a field's expression, that field is considered an aggregate field.

3.6 How to identify whether a field is an aggregate field? #

In addition to determining this through the expression, you can also place the field in the metrics bar. If the word "Aggregate" appears in front of the metric capsule, it is an aggregate field; otherwise, it is a non-aggregate field.

3.7 What does "Sum" mean when it appears in front of a non-aggregate field in the metrics bar? #

Except for detail tables, other charts will automatically apply an aggregation method to non-aggregate fields during queries. For example, the fields shown in the figure below will be automatically aggregated during queries. For numeric types, the default aggregation method is sum, and the value displayed in the chart is the result of summing this field.

The user can click the downward triangle in front of the metric to select another aggregation method.

For aggregate fields, since they have already been aggregated, the word "Aggregate" will appear in front of the metric capsule, and no other aggregation method can be selected.

3.8 Can aggregate fields be placed in dimensions? #

When an aggregate function is used in a hierarchical field (used as a dimension), the LOD function can be used. The LOD function can calculate results aggregated by specified dimensions. Importantly, although aggregation is used within the LOD function, its calculation result is not an aggregate field.
For a detailed introduction to the LOD function, see "LOD Expression"
Example: To categorize provinces into tiers based on the total payment amount for each province, those with amounts greater than or equal to 100,000,000 are tier one, those less than 100,000,000 but greater than or equal to 10,000,000 are tier two, and others are tier three. Finally, the goal is to count the number of provinces in each tier.
Generally, the user might calculate this using the expression shown in the figure below.

However, this field can only be placed in measures, not in dimensions. This is because this expression uses an aggregate function, making it an aggregate field.

At this point, you can replace sum('sale_price') with an LOD function: {filter=true:fixed [sale_province]:sum(sale_price)}

A field created in this way can then be placed in dimensions.

3.9 Field definition issues & whom to ask if field data does not match the online data? #

Background: The DataWind platform does not know what data is stored in the user's dataset, nor does it know the definitions of each field. Only the dataset administrator is aware of these issues.
If a user encounters similar issues while viewing a dashboard, they can ask the dashboard owner.
If a user encounters similar issues during a visualization query, they can first ask the owner of the dataset currently being queried.