Hardware requirements and recommendations for Asset Server

One of the challenges that you will encounter when setting up an Assets Server environment is finding out what hardware is needed.

The best choice of storage, memory and processors depends on various factors.

Minimum requirements

Before discussing the hardware recommendations, please take note of the minimum requirements for running Assets Server:

Outlining your requirements

To find out what requirements your system needs to meet, find the answers to the following questions:

• Users:
• How many users will be using the system at the same time?
• Will users frequently perform searches?

• Assets
• How many files will be stored in Assets Server?
• What type of files will be stored: images, videos, office documents, text files, ...?

• Daily intake
• Will files be frequently imported (manually or automatically)?
• Will there be constant feeds of incoming material?
• Will there be high volume bursts of incoming material?

Use your answers and the information in the following sections to find the hardware required for your environment.

Storage

Storage is required for storing the following files:

• Original assets. The required disk space for storing the original assets depends on their type and their size: text files and XML files will obviously take up far less space than images and videos.
• Index. The required disk space for storing the index depends on the type of assets that are stored: text files, office documents and PDFs are fully indexed and will therefore increase the size of the index. For images and videos only the metadata is indexed which results in a lesser impact on the index size.
• Previews. Previews in Assets Server are typically light-weight but can take up disk space for certain file types such as PDF files and videos (for which the first few minutes are stored as the preview).
• File versions. When files are edited and subsequently checked-in, a new version of the file is generated which can be restored as the main version when needed. This typically takes place for images, layouts and Word documents. Depending on the number and type of assets for which versions are created, additional storage is required.

Example: storage for 1 million assets

Use the following comparison for the amount of disk space needed to store 1 million assets of various types:

Local, fast and large storage

The original assets, previews and search index are stored in a file system. In a small environment, this can be put on a local disk.

For larger environments it can be useful to split the data:

• Place the index (stored in the Hot Data folder) on fast storage when many searches are performed by the users. Typically you would use 10000 RPM or 15000 RPM SAS or SCSI disks, or even SSD disks. Because the index is usually only a few gigabytes in size, this disk does not need a lot of storage capacity.

• Store high numbers of original assets, previews and file versions on large disks or a SAN (Storage Area Network). If you expect that number to grow, use a volume that is expandable such as a SAN solution which makes it possible for extra disks to be added to an existing volume.

SAN or DAS?

For stability and performance reasons we advice to use a storage area network (SAN) or direct-attached storage (DAS).

File Store

The main File Store that contains the original assets cannot be partitioned over multiple disk volumes; spreading data over disks should be handled by the SAN or RAID. For best performance, make sure that all File Stores (assetFilestore, renditionFilestore and tempFilestore) are on the same volume. This way, files are moved instead of copied (the process of copying a file is very slow).

Note: In case you use the NFS protocol it is recommended to use a volume on which the ‘no caching (noac)’ option is enabled. This prevents clients from caching file attributes so that applications can more quickly detect file changes on the server (source: linux.die.net: nfs - Linux Man Page).

Non-supported storage systems

The following storage solutions are not supported for use with Assets Server:

Note: This list is not complete. When in doubt, contact WoodWing.

• Distributed Files System (DFS)

Memory

Memory is used by Assets Server to:

• Cache the search index (a memory cache is used for sorting and facet filtering)
• Run scheduled modules
• Run Web services

Memory consumption increases when:

• The size of the index increases
• Multiple fields are used for sorting and filtering

General guidelines

The following overview shows the recommended amount of memory for various scenarios:

Note: When Assets Server acts as a Search node, 8 GB of memory is the absolute minimum.

CPU cores

The number of CPU cores needed depends mostly on the number of users that are actively using Assets Server.

Hardware examples

The following hardware configurations have proven reliable in performance.

Assets Server

• Mac mini. A Mac mini with a dual-core Intel Core and 8GB of memory is an excellent configuration for a small number of users with a fair amount of assets and a low daily intake of new assets.

It is a simple machine to install with Mac OS X client or Mac OS X Server (in case you want to run an LDAP service). The hardware can be configured with 1 or 2 hard disks (OS X server version only), possibly in a RAID configuration and/or using an external hard disk for storage.

• Mac Pro. A Mac Pro with a Quad-Core Intel Xeon processor and 8–16GB of memory is very suitable for a small to average number of users with many assets and a low daily intake of new assets. The extra number of cores and processors makes this a much more powerful configuration than the Mac mini. Because of the multiple hard disk bays it is also possible to set up a RAID or SoftRAID configuration or use SSD storage for storing the index for exceptional search performance. Optionally, SAN solutions can dramatically expand the storage capacity if needed.

By adding extra Mac minis or Mac Pros as Assets Server Processing nodes it is very easy to expand processing power to handle more assets, thereby allowing the Search node to handle more users.

• Windows system. A Windows Server with a Quad-Core Intel Xeon processor (or faster) and 16–32GB of memory is very suitable for a small to average number of users and a high number of assets with an average daily intake of new assets. Advised are machines with multiple hard disk bays so that it is possible to set up a RAID or SoftRAID configuration. This also offers the option to use SSD storage for storing the index for exceptional search performance. Optionally, SAN solutions can dramatically expand the storage capacity if needed.

Processing node

Spreading the server load

When content is added to the system, Assets Server generates thumbnails and previews, and extracts text and metadata. Processing these tasks requires a lot of CPU power and memory. Performance can be considerably improved by setting up one or more dedicated Processing nodes.

Figure: A typical cluster consisting of 3 Search nodes and 2 Processing nodes.

For more information, see The Assets Server structure: clusters and nodes.

Connecting the Search node to the Processing node

We advise to connect the Search nodes and the Processing nodes to the same physical switch and in the same network subnet. If there are too many network switches or routers in between, the connection might become unstable and performance will degrade because of the large amounts of data that is sent back and forth.

Daily intake

How many processing capacity you need depends mostly on how many files are imported per day and how heavy those files are to process. For example, if you have a number of wire feeds being imported by scheduled folder-import modules, you may need additional Processing nodes. Video files and high-res material can take longer to process, so if you expect heavy material to be imported on a regular base, use more or heavier Processing nodes.

Redundancy

Since you can use a cluster of multiple Processing nodes, it makes sense to use at least two or more (a typical cluster consists of 3 Search nodes and 2 Processing nodes). Doing so provides not only increased processing power, but also provides automatic redundancy when one of the processing machines fails. Assets Server will detect a processing node that is no longer responding and will redirect any processing requests to the remaining machines in the cluster.

For more information, see The Assets Server structure: clusters and nodes.

Processing server hardware examples

Typical hardware used for processing servers:

Mac mini cluster

• Slower than a Mac Pro
• Cheap

You can build a cluster of about 3 Mac mini's for the same price as one Mac Pro. In some cases this can result in higher total throughput. Be aware though that each individual file will take longer to process.

When using Mac mini's you will typically use at least two or more of them. You do not necessarily need the 'server' version of the Mac mini. A simple desktop model will do: no RAID configuration is required and redundancy is already provided because you are using more than one Processing node.

The advantage of a server model is that its CPU is faster and the hard drives are faster (when configured as a maximum performance RAID set).

Mac Pro (cluster)

• High-performance Input-Output system
• Much faster than a Mac mini
• More expensive than a Mac mini

Very suitable when many files are imported into Assets Server and previews need to be available as fast as possible.

Although they require more rack-space than Mac mini's, these machines are very powerful. You will need quite a few Mac mini's to get the same capacity as one or two Mac Pros.