Updating the file index of Assets Server

When users search for a file, Assets Server uses an index to locate where the file is stored. It is important therefore that the index is always up-to-date.

Available indexes and when to update them

Assets Server contains multiple indexes but the only one that you will ever typically update is the 'file' index.

Note: Updating the file index is mainly necessary when: Changing custom metadata in the custom-assetinfo.xml file You are prompted by Assets Server to do so

Other available indexes include:

• config
• Enterprise-updates
• import
• processing
• relation
• security
• settings
• stats
• users
• various
• version

Whenever an index needs to be manually updated, this will be mentioned in a Help Center article.

Updating the index

Note: Updating an index might take a while, depending on the size of your index.

Access the Management Console and choose Tools > Index Revision Tool in the menu on the left side of the page.

To update an index, click its Re-index button.

Use the Pause and Continue buttons to temporarily stop the process (requires Assets Server 6.60 or higher). Use this when indexing takes a long time and the performance of the client applications is affected by it; pause it during working hours and resume it in between working hours.

When the indexing succeeds, the revision number will be incremented and your new configuration will have taken effect. If it fails, you will get an error message and no changes will have been made to the index.

Notes: See the comments at the end of this article for more information. In Assets Server 6.112 and higher, a Documents column is available. It shows the total number of documents that are part of the index. During re-indexing, it shows the number and percentage of the documents that have been re-indexed so far. In Assets Server 6.112 and higher, the state of the buttons is automatically updated when the re-indexing process finishes or fails,

Optimizing the indexing performance

The indexing performance of Elasticsearch can be optimized by disabling its merge throttling during the indexing process. This will speed up the process of indexing significantly. Throttling can be enabled again when the indexing is completed.

Notes: Only set this property if users have no access to the system at the time, as it will have a high impact on performance. These steps can be applied while the re-indexing process is ongoing.

Disabling throttling is done by setting the indices.store.throttle.type option to 'none': 

Copy the command

curl -XPUT 'localhost:9200/_cluster/settings' -d '{ "transient" : { "indices.store.throttle.type": "none" }}'

The throttle setting will reset after a full restart of the cluster as it is transient. To avoid the need of a restart, it can be reset manually to the original value:

Copy the command

curl -XPUT 'localhost:9200/_cluster/settings' -d '{ "transient" : { "indices.store.throttle.type": "merge" }}'

Troubleshooting

Why is the config index always set to '1' after restarting the server?

At each server restart, the config index is regenerated and redistributed across all nodes and will therefore always be '1', even when initially incrementing it by updating the index.

Errors appear in the log files after updating the asset index

The following errors might appear in the log files:

ERROR com.ds.acm.ak - NEW CONFIGURATION IS INCORRECT: Compass settings of the field language were changed. It is incorrect to modify the field.

WARN com.ds.acm.warnings.Server - warning: Detected illegal changes in new asset info

com.ds.acm.al: New configuration is incorrect, see log for details. Please review asset info configuration.

These errors are generated by a safety feature that is in place during the Assets Server service startup which compares the full-assetInfo.xml file (created during the last successful startup) with the new-assetInfo.xml file (constructed from the (internal)default-assetinfo file and the possible (custom) configured custom-assetinfo file).

If the compare results in a difference, the above errors are thrown in the Assets Server log with a description where this difference is found.

This safety feature is in place to protect the system from starting with an incorrect assetinfo file.

A common scenario for this situation to occur is when changes have been made to the configuration of the assetInfo in the custom-assetinfo.xml file. It may also occur after upgrading Assets Server to a newer version with an updated default assetinfo.

It is important to be aware if recent changes have been made to your custom-assetInfo.xml:

• Yes, I just updated the configuration of the custom-assetInfo file which resulted in this log message. Please read the log message carefully; it might describe where a possible configuration error occurred.

• No, I just upgraded my Assets Server version (with a new/changed default assetinfo file) which resulted in this log message.

Solution 1

Update the asset index once more.

Solution 2

Info: Performing the described step requires direct server access. Depending upon how your system is hosted and the level of access you have to that system, coordination may be required with your Partner or WoodWing Support team. For a full overview of the steps that need to be done by WoodWing and how to request them, see WoodWing Cloud - Change management.

Do the following:

Step 1. Stop your Assets Server Cluster nodes.

Step 2. On each node, access the <Assets Server location>/Elvis Hot Data/elvis-data/assetInfo/ folder and remove the following files:

• full_assetinfo.xml
• new_assetinfo.xml

Note: Make sure you do this on every node (including the nodes that act as the Processing Server and job runner).

Step 3. Start the Assets Server Cluster.

New files will be created and the errors should not appear anymore.

Revisions

• 17 September 2024: Updated the notes in section 'Updating the index' with information about improvements made in Assets Server 6.112.