Custom stats records can be stored in the Assets Server stats index through the logUsage API. The metadata that is stored in these stats records can also contain custom metadata.
Searching and especially aggregating (reporting) on this data can be very useful to perform reporting on the data stored in the stats index.
This article describes how to use the logUsage API to properly store your custom data.
Examples of querying or aggregating on stats data can be found in the source files of the Assets Server stats project.
Field type mapping
Defining how the field should be indexed by Assets Server is done by specifying a postfix in the following format:
field_name__xx
Where 'xx' points to a specific analyzer with which the data is analyzed. (For an overview of the postfixes that can be used, see below.)
Example: Assume you want to store statistics information about where and when an asset was used (published). In a report, you want to show how many assets where published in a certain period. To make such a report, you would have to store the date on which an asset was published so that you can later on search on a period. To achieve this, the field is configured with the postfix __dt ('dt' stands for 'date/time', see supported field type mappings below). The logUsage API call would be as follows:
This indexes a new stats record with a custom metadata field called 'published_dt'. The field is indexed as a date field as it uses the '__dt' postfix. There is no additional configuration needed here, the type definition is configured automatically with the first logUsage call where the field is provided. |
Important: As mapping types are fixed, it is advised to consider beforehand how your metadata should be mapped before indexing any stats data. Once you data is indexed for a field for the first time, the type mapping is configured and cannot be changed anymore. When data is indexed in a field which cannot be analyzed, the stats record entry will be ignored and an error is logged in the Assets Server log file.
Supported field type mappings
The following overview shows all supported mapping types and a description of their behavior.
Note: Elasticsearch supports a wide range of query types; it is out of scope to discuss them here in detail. The examples are limited to term, range and match queries.
default (no postfix)
- Description: When there is no postfix specified, Assets Server will interpret the field as a standard text field. This means the text is tokenized into separate tokens (words or part of words) using the Elasticsearch English analyzer. Querying is possible on individual tokens.
- Example values: "The quick brown fox jumps over the lazy dog"
- Elasticsearch query examples:
Example 1: This term query will find the example value:
Other term queries that work:
|
Example 2: This match query will find the example value:
Other match queries that work:
Queries that will not work:
|
Date-time
- Postfix: __dt
- Description: Date and date-time field. Supported date-time formats are:
yyyy-MM-dd'T'HH:mm:ss.SSSZ
yyyy-MM-dd'T'HH:mm:ssZ
yyyy-MM-dd'T'HH:mm:ss
yyyy-MM-dd HH:mm:ss.SSS Z
yyyy-MM-dd HH:mm:ss.SSS
yyyy-MM-dd HH:mm:ss Z
yyyy-MM-dd HH:mm:ss
yyyy-MM-dd
yyyy/MM/dd HH:mm:ss.SSS Z
yyyy/MM/dd HH:mm:ss.SSS
yyyy/MM/dd HH:mm:ss Z
yyyy/MM/dd HH:mm:ss
yyyy/MM/dd
HH:mm:ss.SSS
HH:mm:ss
- Example values:
2018-06-24T12:34:56.543-0800
2018-06-24T12:34:56-0800
2018-06-24T12:34:56
2018-06-24 12:34:56.543 -0800
2018-06-24 12:34:56.543
2018-06-24 12:34:56 -0800
2018-06-24 12:34:56
2018-06-24
2018/06/24 12:34:56.543 -0800
2018/06/24 12:34:56.543
2018/06/24 12:34:56 -0800
2018/06/24 12:34:56
2018/06/24
12:34:56.543
12:34:56
- Elasticsearch query examples:
Example: Find all stats record between January 1st 2018 and October 31st 2018:
|
Number
- Postfix: __nr
- Description: Number field.
- Example values: 123456789012345
- Elasticsearch query examples:
Example: Find all stats records with value 123456789012345:
|
Decimal
- Postfix: __dc
- Description: Decimal field.
- Example values: 12345.678
- Elasticsearch query examples:
Example: Find all stats records with value greater than 50"
|
Boolean
- Postfix: __b
- Description: Boolean field.
- Example values:
true
false
0
1
- Elasticsearch query examples:
Example: Find all stats records with value true:
|
Exact text
- Postfix: __s
- Description: Exact text field. The text is not analyzed and not tokenized. You can only query for the exact value that was stored.
- Example values: "The quick brown fox jumps over the lazy dog"
- Elasticsearch query examples:
This term or match query will find the example value:
"The quick brown fox jumps over the lazy dog"
Queries that will not work:
"The quick brown FOX jumps over the lazy dog"
"fox"
Case insensitive text
- Postfix: __si
- Description: Case insensitive text field. Same as exact text __s, with the addition that queries can be case insensitive.
- Example values: "The quick brown fox jumps over the lazy dog"
- Elasticsearch query examples:
This term query will find the example value:
"the quick brown fox jumps over the lazy dog"
This match query will find the example value:
"The quick brown FOX jumps over the lazy dog"
Queries that will not work:
" the quick brown fox jumps over the lazy dog"
"The quick brown fox"
Trimmed and case insensitive text
- Postfix: __st
- Description: Trimmed and case insensitive text field. Same as case insensitive __si, with the addition that the text is trimmed, ignoring any white space at the beginning and end of the text when querying.
- Example values: " The quick brown fox jumps over the lazy dog"
- Elasticsearch query examples:
This term query will find the example value:
"the quick brown fox jumps over the lazy dog"
These match queries will find the example value:
"The quick brown fox jumps over the lazy dog"
" The quick brown fox jumps over the lazy dog"
Queries that will not work:
"The quick brown fox"
Case insensitive and tokenized
- Postfix: __sa
- Description: Case insensitive and tokenized text field that divides text at numbers, punctuation and other non-alphabetic characters.
- Example values: "The 25 quick brown fox3s jumped over the lazy dog"
- Elasticsearch query examples:
These term queries will find the example value:
"brown"
"fox"
These match queries will find the example value:
"The 25 quick brown fox3s jumped over the lazy dog"
"fox3s"
Queries that will not work:
"25"
Case insensitive and tokenized (divided at path delimiter)
- Postfix: __sh
- Description: Case insensitive and tokenized text field that divides on a path delimiter: "/". Used for searching on part of a path or on path ancestors.
- Example values: "/This is/a typical/path"
- Elasticsearch query examples:
These term queries will find the example value:
"/this is"
"/this is/a typical"
"/this is/a typical path"
These match queries will find the example value:
"/This is/a typical/path"
"/this is/a typical/path"
Queries that will not work:
"/a typical/path"
Default but with an explicit language
- Postfix: __s_xx
- Description: Same as default but with an explicit language. For example __s_nl maps to Dutch and __s_ja maps to Japanese.
Supported languages:
ar - Arabic
ca - Catalan
cs - Czech
da - Danish
de - German
el - Greek (modern)
en - English
es - Spanish
fi - Finnish
fr - French
ga - Irish
hi - Hindi
hu - Hungarian
in - Indonesian
it - Italian
ko - Korean
lv - Latvian
nl - Dutch
no - Norwegian
pl - Polish
pt - Portuguese
ro - Romanian
ru - Russian
sv - Slovak
th - Thai
tr - Turkish
zh - Chinese
Comment
Do you have corrections or additional information about this article? Leave a comment! Do you have a question about what is described in this article? Please contact Support.
0 comments
Please sign in to leave a comment.