Using S3 as archive storage when running Assets Server in a non-Amazon environment

When running Assets Server in a non-Amazon environment (such as a local office or a data center) it is possible to configure S3 as a storage engine for your archive of infrequently used files.

This article explains how to do this.

Note: For frequently used production files a local shared-volume storage engine should be used.

Configuration considerations

Consider the network speed between the Assets Server installation and Amazon S3 and how this relates to the speed of uploading and downloading assets:

• A normal or slow connection to AWS is suitable only for storing archived files that are accessed on occasion.
• A really fast connection (for example using Amazon Direct Connect) might make it possible to store highly requested production files. (Based on our current knowledge, none of our customers are using S3 storage in this way.)

Setup and configuration

The process involves setting up AWS and configuring Assets Server.

1. Setting up AWS

Step 1. Set up an S3 bucket for use by Assets Server.

See the AWS documentation: Getting Started with Amazon Simple Storage Service.

Step 2. Set up an IAM user with Programmatic access and an access key. Grant it permission to access the bucket.

See the AWS documentation:

• Creating IAM Users (Console)
• Amazon S3: Allows Read and Write Access to a Specific S3 Bucket

2. Configuring Assets Server

Add an archive S3 storage engine. Make sure that the /Archive folder in Assets Server does not exist yet, or at least does not contain any assets yet. After the configuration has been set up and all Assets Server nodes have been restarted, assets can be moved into the /Archive folder in Assets Server.

Configure the storage-engine-config.conf file (the configuration will typically be identical on all nodes):

Copy the code

storage.engine {
  type = "multi-tier"
  default-engine {
    type = "${storage.engine.type}"
    sharedDataLocation = "${sharedDataLocation}"
    assetFilestore = "${assetFilestoreLocation}"
    renditionFilestore = "${renditionFilestoreLocation}"
    tempFilestore = "${tempFilestoreLocation}"
    mounted = ${mounted} # from the `filestoreType` legacy config option (`shared`=true,`local`=false)
  }
  engines : [
    {
      name = "archive"
      type = "s3"
      pattern = "/archive/**"
      region = "eu-west-1"
      bucket = "assets-archive"
      accessKeyId = ""
      secretKey = ""
      #tempDirLocation = "/var/tmp/asets_server"
    }
  ]
}