WoodWing Help Center

The Elvis 6 structure: clusters and nodes

The Elvis 6 structure: clusters and nodes

Elvis 6 is a dynamically scalable system with a high level of built-in availability, redundancy and load balancing.

The Elvis 6 structure consists of a cluster of one or more nodes, each with the ability to take on a specific role such as that of a search engine for searching and indexing, a processing engine for processing files and generating previews, or a job scheduler for running scheduled jobs.

Note: Technically, each node is the same and each is installed by using the same installer. Its role is determined by changing the configuration during the installation.

By comparing the differences with the Elvis 4 structure, the advantages of the Elvis 6 structure become clear:

Elvis 4 Elvis 6
Complete search index on one machine

Each search node holds part of the index

Full search load on one machine

All nodes share the search load

All clients talk to one machine Clients talk to any Web API node, load is balanced
Processing nodes access storage through Main Server All nodes have direct access to the storage
Only the Processing Server can be run separately Multiple nodes can be set up with different roles

Scaling is done vertically and is therefore limited

Scaling is done horizontally, allowing storage of billions of assets

Elvis 4 and 5 structure comparison

Figure: An Elvis 4 setup (left) and an Elvis 6 medium cluster (right).

Search engine

The search engine that is used in Elvis 6 is that of elasticsearch. It takes care of the horizontal scaling, searching and indexing.

Communication between nodes

The communication between nodes is done by making use of Hazelcast technology.

Cluster examples

Because of its distributed nature, possible clusters range from including just a single node to including as many nodes as needed to support millions or billions of files.

Note: A single node can typically already support a few million files.

This article outlines some typical clusters.

Notes about the numbers given in the examples:

  • The actual number of users that can be served depends on how heavy they will use Elvis and what kind of operations they perform.
  • The actual maximum users and maximum assets depends not just on the number of nodes but also on the kind of hardware used for those nodes.

The following table provides a summary:

  Single machine Small cluster Medium cluster Big cluster
Number of nodes 1 3 5 Dozens
Maximum number of concurrent users 10 – 20 20 – 40 50 – 100 5,000+
Maximum number of assets 5 million 5 million 50 million 1 billion
Daily asset intake Low volume Medium volume Medium volume High volume
Data redundancy? No No 1 1 Search node is allowed to fail without data loss 2 Several Search nodes are allowed to fail without data loss 1
Processing redundancy? No 1 processing node is allowed to fail 1 Processing node is allowed to temporarily fail Several Processing nodes are allowed to fail 3

1 It is assumed here that the 3-node cluster consists of 1 search node and 2 processing nodes.

2 It is assumed here that the 5-node cluster consists of 3 search nodes and 2 processing nodes.

3 When several nodes fail, performance will be affected; a new node is expected to take the place of a failed node.

A single server of 1 node

A single-server setup

In this cluster, Elvis Server is installed on a single machine and therefore takes on all roles.

This cluster is not redundant: when the machine is down, Elvis is not available.

Use this cluster for a maximum number of 10 – 20 users and for handling up to 500,000 assets.

A small cluster of 3 nodes

A cluster with 3 nodes

In this cluster, 1 Elvis installation is used as a Search node while the load of processing files and generating previews is balanced over 2 separate Elvis installations that act as Processing nodes.

This cluster is semi-redundant: 1 Processing node is allowed to fail. However, make sure that it is brought back up (or replaced) as soon as possible.

Use this cluster for a maximum number of 20 – 40 users and for handling up to 5 million assets.

A medium cluster of 5 nodes

This is a typical cluster and consists of 3 Search nodes and 2 Processing nodes.

It can be expanded as needed by adding more Search nodes or Processing nodes. There does not necessarily have to be a constant factor between the number of nodes.

This cluster is fully redundant and resilient: when the cluster contains at least 3 Search nodes, Elvis will automatically enable 1 replica shard for each index.

If 1 of the Search nodes goes down, the other 2 will keep running and all data will still be available. However, make sure that the other Search node is brought back up (or replaced) as soon as possible. When one of the Processing nodes dies the other one will take on all processing. Again make sure to replace the broken Processing node.

Note: To be completely resilient, a load balancer needs to be added in front of the Elvis nodes that need to be used as web API front-ends. This also helps to spread the load when many concurrent users are using the system.

Use this cluster for a maximum number of 50 – 100 users and for handling up to 50 million assets.

A big cluster

A cluster of many nodes

Scaling up an Elvis 6 cluster is limitless, as this theoretical example shows in which "dedicated Master nodes" prevent heavy load from influencing cluster stability.

This cluster is fully redundant: several Search nodes are allowed to fail without data loss and several Processing nodes are allowed to fail.

Such a cluster would be used for environments with more than 5,000 users, handling up to 1 billion assets.

Note: Huge deployments are possible with the right configuration and topology. As this requires extensive knowledge of the system, please contact support@woodwing.com.

Was this article helpful?
0 out of 0 found this helpful / Created: / Updated:
Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.