Running Assets Server in the Amazon Web Services (AWS) cloud

When running Assets Server within Amazon Web Services (AWS) it is possible to use S3 as a storage engine.

Configuration considerations

When using Amazon S3 as a storage engine for Assets Server, please take note of the following:

• When running the complete Assets Server installation on AWS, Amazon S3 is the recommended storage solution. As well as being cost-effective, it ensures availability and proper performance.

• When running Assets Server together with a WoodWing Studio Server integration and both are set up on AWS, please also view Integrating Assets Server with Studio Server on AWS - Load balancer information.

• When running the Assets Server installation in a non-Amazon environment (such as a local office or a data center), consider the network speed between the Assets Server installation and Amazon S3 and how this relates to the speed of uploading and downloading assets:
• A slow connection to AWS is suitable only for storing archived files that are accessed on occasion
• A fast connection makes storing highly requested production files possible. See Using S3 as archive storage when running Assets Server in a non-Amazon environment.

• When using Amazon S3 as a storage engine, be aware that the feature that automatically creates an Elasticsearch backup by default references sharedDataLocation in its setting elasticsearch.backup.location. If the Search nodes will not have a file system based shared data location, the backup should be disabled. Instead, you can manually configure and manage an S3-based backup repository in Elasticsearch.

• Assets Server only supports Instance Metadata Service version 1 (IMDSv1). IMDSv2 is not supported. Configuring your S3 storage to use IMDSv2 will prevent Assets Server from connecting to the S3 storage.

AWS setup

Note: Experience in working with AWS is needed to determine the correct setup, depending on what your requirements are. We strongly advise to use an infrastructure management tool such as AWS CloudFormation or Terraform to create, update, manage, and document your deployment.

Ports and Security Group

Ensure that the Security Group for the Assets Server nodes has the proper TCP port configuration:

• 5701, only accessible from the Assets Server nodes Security Group itself.

• 9300, only accessible from the Assets Server nodes Security Group itself.

• 80, accessible from the Security Group of the Load Balancer. This assumes that HTTPS termination on the Load Balancer is used.

The Security Group name is also used for discovery between the Assets Server nodes, see the cluster.join.aws.securityGroupName configuration property mentioned below.

IAM Role and Policy

Set up an IAM Role that grants permissions for the EC2 instances that are running Assets Server. The role should be assigned to the EC2 instance(s). When running on an EC2 instance, Assets Server will automatically assume that role to connect to S3 and other AWS services.

At least the following IAM policies must be configured for the Role.

Grant access to S3:

Copy the code

{
 "Effect": "Allow",
 "Action": [
   "s3:PutObject",
   "s3:GetObject",
   "s3:DeleteObject",
   "s3:ListBucket"
 ],
 "Resource": [
   "arn:aws:s3:::your-storage-bucket/*",
   "arn:aws:s3:::your-storage-bucket"
 ]
}

Allow the nodes to query the AWS environment to find other Assets Server nodes with the same Security Group and form a cluster:

Copy the code

{
  "Effect": "Allow",
  "Action": [
    "ec2:DescribeInstances"
  ],
  "Resource": [
    "*"
  ]
}

Load Balancer

We advise using an EC2 Application Load Balancer if you need one. Make sure that sticky sessions are turned ON, this will improve cache efficiency within Assets Server.

Configuring Assets Server

On each node, configure specific Assets Server S3 and AWS related configuration properties.

Step 1. Configure the node-config.properties.txt file:

Copy the code

# Since IPs are typically dynamic on EC2, the tcpip.members method does not work very well.
cluster.join.tcpip.enabled=false
  
# Instead, we use the aws join method. Specify the correct region and the name of the SecurityGroup assigned to the Assets Server nodes.
cluster.join.aws.enabled=true
cluster.join.aws.region=eu-west-1
cluster.join.aws.securityGroupName=sg-...
 
# Just used for S3 temp file storage, use fast local storage
sharedDataLocation=
 
# Set it to local, otherwise it will wait until shared-data is mounted
fileStoreType=local

Step 2. Also add the following to the node-config.properties.txt file:

Copy the code

# Set storage engine type to S3
storage.engine.type=S3
# The name of your S3 bucket
storage.engine.s3.bucket=
# Region of the S3 bucket e.g. eu-west-1
storage.engine.s3.region=