Elvis 6 is a dynamically scalable system with a high level of built-in availability, redundancy and load balancing.
The Elvis 6 structure consists of a cluster of one or more nodes, each with the ability to take on a specific role such as that of a search engine for searching and indexing, a processing engine for processing files and generating previews, or a job scheduler for running scheduled jobs.
Note: Technically, each node is the same and each is installed by using the same installer. Its role is determined by changing the configuration during the installation.
By comparing the differences with the Elvis 4 structure, the advantages of the Elvis 6 structure become clear:
| Elvis 4 | Elvis 6 |
|---|---|
| Complete search index on one machine |
Each search node holds part of the index |
| Full search load on one machine |
All nodes share the search load |
| All clients talk to one machine | Clients talk to any Web API node, load is balanced |
| Processing nodes access storage through Main Server | All nodes have direct access to the storage |
| Only the Processing Server can be run separately | Multiple nodes can be set up with different roles |
|
Scaling is done vertically and is therefore limited |
Scaling is done horizontally, allowing storage of billions of assets |
Figure: An Elvis 4 setup (left) and an Elvis 6 medium cluster (right).
Search engine
The search engine that is used in Elvis 6 is that of elasticsearch. It takes care of the horizontal scaling, searching and indexing.
Communication between nodes
The communication between nodes is done by making use of Hazelcast technology.
Cluster examples
Because of its distributed nature, possible clusters range from including just a single node to including as many nodes as needed to support millions or billions of files.
Note: A single node can typically already support a few million files.
This article outlines some typical clusters.
|
Notes about the examples:
|
The following table provides a summary:
| Single machine | Small cluster | Medium cluster | Big cluster | |
|---|---|---|---|---|
| Number of nodes | 1 | 3 | 5 | Dozens |
| Maximum number of concurrent users | 10 – 20 | 20 – 40 | 50 – 100 | 5,000+ |
| Maximum number of assets | 5 million | 5 million | 50 million | 1 billion |
| Daily asset intake | Low volume | Medium volume | Medium volume | High volume |
| Data redundancy? | No | No 1 | 1 Search node is allowed to fail without data loss 2 | Several Search nodes are allowed to fail without data loss 1 |
| Processing redundancy? | No | 1 processing node is allowed to fail | 1 Processing node is allowed to temporarily fail | Several Processing nodes are allowed to fail 3 |
1 It is assumed here that the 3-node cluster consists of 1 search node and 2 processing nodes.
2 It is assumed here that the 5-node cluster consists of 3 search nodes and 2 processing nodes.
3 When several nodes fail, performance will be affected; a new node is expected to take the place of a failed node.
A single server of 1 node
In this cluster, Elvis Server is installed on a single machine and therefore takes on all roles.
This cluster is not redundant: when the machine is down, Elvis is not available.
Use this cluster for a maximum number of 10 – 20 users and for handling up to 5 million assets.
A small cluster of 3 nodes
In this cluster, 1 Elvis installation is used as a Search node while the load of processing files and generating previews is balanced over 2 separate Elvis installations that act as Processing nodes.
This cluster is semi-redundant: 1 Processing node is allowed to fail. However, make sure that it is brought back up (or replaced) as soon as possible.
Use this cluster for a maximum number of 20 – 40 users and for handling up to 5 million assets.
A medium cluster of 5 nodes
This is a typical cluster and consists of 3 Search nodes and 2 Processing nodes.
It can be expanded as needed by adding more Search nodes or Processing nodes. There does not necessarily have to be a constant factor between the number of nodes.
This cluster is fully redundant and resilient: when the cluster contains at least 3 Search nodes, Elvis will automatically enable 1 replica shard for each index.
If 1 of the Search nodes goes down, the other 2 will keep running and all data will still be available. However, make sure that the other Search node is brought back up (or replaced) as soon as possible. When one of the Processing nodes dies the other one will take on all processing. Again make sure to replace the broken Processing node.
Note: To be completely resilient, a load balancer needs to be added in front of the Elvis nodes that need to be used as web API front-ends. This also helps to spread the load when many concurrent users are using the system.
Use this cluster for a maximum number of 50 – 100 users and for handling up to 50 million assets.
Large clusters
Large clusters typically involve adding load balancers and dividing clusters over multiple data centers.
For implementing such large clusters, please contact WoodWing Support to discuss the expectations, setup, and planning.
Comment
Do you have corrections or additional information about this article? Leave a comment! Do you have a question about what is described in this article? Please contact Support.
2 comments
Please add an example of ‘Big cluster’ with ‘Dozens’ of nodes, including 'Job Runner’ nodes and load balancer.
In the article, it states: "It can be expanded as needed by adding more Search nodes or Processing nodes. There does not necessarily have to be a constant factor between the number of nodes."
If I have a cluster of 5 nodes (3 Search / 2 Processing) and automatically increase it to 6 nodes (4 Search / 2 processing), do I need to update the cluster config as mentioned in the document below before scaling takes place?
https://helpcenter.woodwing.com/hc/en-us/articles/115002615463#3.-basic-server-configuration
I can imagine a setup where we use AWS Load Balancing which can auto-scale based on CPU/Memory. However, if we need to manually update the cluster config in Elvis, it might need some extra work.
Is my understanding correct? Thanks!
Please sign in to leave a comment.