Working with Enterprise Server Jobs in Enterprise Server 9

About Enterprise Server Jobs

An Enterprise Server Job is a task that is performed by Enterprise Server, such as clearing all objects of the Trash Can at set times or processing images during an import or export process.

This task does not necessarily have to be performed on the workflow machine: it can also be delegated to a different instance of Enterprise Server. Possible reasons for doing this include:

• Running the process is expected to take too long on the workflow machine, which would result in end users having to wait for the process to complete.
• Running the process is expected to consume too many resources (memory) on that machine, possibly causing memory swapping of other processes servicing other users, resulting in unpredictable performance.
• The workflow machine is not able to perform the task because the required tools (such as Image Magick, Tika, and so on) are not available.

Other scenarios in which Server Jobs are useful include:

• The process requires more access rights than the acting user has, and therefore there is a need to configure a user with more access rights for the job.
• The process has an administrative purpose and needs to be repeated periodically. The job can then be scheduled to run at certain times.

In all instances, Server Jobs are queued and processed asynchronously in the background. End-users therefore do not have to wait for Enterprise Server to process a job and are not affected by heavy processing or memory consumption. These are the main goals of the Server Jobs concept.

Terminology

When working with Server Jobs, the following terminology is used:

Scheduled job

A job scheduled to be repeated at set intervals. For example: clearing a temp folder every day.

User-initiated job

A job triggered by an end-user. For example: uploading an image, which results in generating a preview of that image.

Server Job Queue

Server Jobs are placed in the Server Job Queue from which they can be picked up by any Enterprise Server machine.

Server Job configuration

The settings that define a Server Job, including:

• The time interval of a recurring Server Job
• How often a recurring Server Job should run
• The number of attempts that a failed process should be retried

Background versus foreground processing

From an Enterprise Server point of view, the process of running a Server Job is done in the foreground; however, from an end-user’s point of view, the process is run in the background. This allows the users to continue their work without having to wait for the process to be completed first.

Enterprise Server farm

A cluster of Enterprise Servers working together.

Server Job Processor

The Server Job Processor is the part of the system that picks up available Server Jobs from the Server Job Queue.

Co-worker machine

A Server machine running Enterprise Server that is dedicated to pick-up some or all jobs from the queue.

Note: By design, a Co-worker machine is not accessible by client applications and therefore does not act as a Workflow Server. Only under specific circumstances (for testing, demos, development, or even small offices) should a Co-worker machine be made available to client applications; doing so introduces the old issues of unpredictable performance due to heavy workload on the system.

Workflow Server

An instance of Enterprise Server set up to offer the workflow features (assigning a status, routing to users, and so on) to client applications.

Workflow Server versus Co-worker machines

There is no difference between installing Enterprise as a Workflow Server or as a Co-worker machine; each Enterprise Server system can be configured to run as a Co-worker, a Workflow Server, or both:

• When Enterprise Server is not configured to pick up any Server Jobs, it is automatically considered to be a dedicated Workflow server.
• When Enterprise Server is configured to pick up some or all Server Jobs, it is automatically considered to be a Co-worker machine.

Scheduler

A Scheduler is a continuously running engine that drives the Server Jobs mechanism for an Enterprise Server farm. It manages the following tasks:

• Continuous processing of jobs from the queue.
• Periodic creation of jobs (instances) based on recurring job configurations.

How Server Jobs are processed

Once a Server Job is created, it needs to be picked up. This can be done by each instance of Enterprise Server and is performed by the Server Job Processor: a PHP server module triggered by a scheduler.

For each process, the scheduler starts a new Job Processor. Each process runs for a very limited time (for instance a few minutes), after which the process no longer picks up new jobs from the queue and ends by itself. During the time the process runs, the scheduler creates new processes, thereby creating a pool of processes.

Idle times

When no jobs are available to be run, one process will be connected to watch for jobs arriving in the queue. This way, very low resources are used, enabling the whole server machine to become idle. When the waiting process picks up a job that just arrived from the queue, it gives way for other processes to come into action again to monitor and process the queue.

High workloads

When more jobs are available than the system can currently handle, the Server Job Processor will keep these processes on hold to allow the jobs being processed to complete first.

Reattempts

In case a job fails to be processed, the system tries to understand the reason for the failure and decides if the job can be re-run. This can be repeated a set number of times after which the process is removed.

Setting up and configuring Enterprise Server Jobs

The following steps are to be performed:

1. Setting up Enterprise Server instances

2. Configuring existing Server Jobs

3. Installing a Scheduler

4. Setting up the job queue processor

5. (Optional) Setting up recurring jobs

6. Validating the installation

1. Setting up Enterprise Server instances

Each individual Enterprise Server can play several roles:

• Application Server (Workflow Server)
• File transfer Server
• Co-worker (background processor)

Some servers in an Enterprise Server farm can have multiple roles, other servers can be clustered with an Apache dispatcher, while yet other servers might be dedicated to a certain role.

Regardless of the intended server role, for each different URL through which those servers (or clusters) can be reached, follow these steps:

Step 1. In Enterprise Server, access the Enterprise Servers page.

Step 2. Click New Server.

The Enterprise Server Maintenance page appears.

Step 3. In the Name box, add a descriptive name.

Step 4. From the Type list, choose Enterprise.

Step 5. In the URL field, type the URL of the server as seen from the client applications.

Example: http://127.0.0.1/Enterprise

Step 6. (Optional) In the Description field, add a description for the server.

Step 7. From the Handle Jobs list, choose All.

Step 8. Click Update.

The Enterprise Servers page appears.

2. Configuring existing Server Jobs

The following Server Jobs are available by default:

• TransferServerCleanUp. For clearing all files from the File Transfer folder.
• AutoPurgeTrashCan. For permanently deleting all files from the Trash Can.

Note: The email address of this user is used to report the results of the Auto Purge operation by e-mail to that user. The e-mail shows how many objects there are/were in the Trash Can, how long the operation took, and any errors that occurred. For information about customizing the e-mail, see Customizing notification e-mails in Enterprise Server 10.

• AdobeDPS. For clearing up the folder to which Adobe DPS publications are exported to from Content Station.

• AutoCleanServerJobs. For clearing up the Enterprise Server Jobs Queue. When using this Server Job, please see Deleting Enterprise Server Jobs from the queue in Enterprise Server 10 for additional configuration steps.

To make these operational, follow these steps:

Step 1. Access the Enterprise Server Job Config page.

Step 2. Click the name of each job and set its options:

• Attempts: Enter a number which specifies how many attempts should be made to re-try the job until running the job should be canceled.
• Username: Choose a user with system admin privileges.
• Active: Select to activate this job.

Step 3. Click Update.

3. Installing a Scheduler

A Scheduler should be set up on every Enterprise Server instance that needs to act as a Co-worker machine. This enables the Co-workers to pick up (heavy) jobs from the queue.

Step 1. Make sure that Enterprise Server is accessible from the command line, for example through the use of cURL.

Step 2. Use a scheduler of your choice.

• CronniX for Mac OS X. CronniX is a GUI for Crontab and makes its configuration a little easier. Crontab is pre-installed on Mac OS X. CronniX can be downloaded from http://www.macupdate.com/app/mac/7486/cronnix.

Note: Alternatively, you can skip the CronniX installation and use Crontab directly, but then please follow the configuration instructions written for Linux.

• Task Scheduler for Windows. Task Scheduler is pre-installed for Windows. No further action is therefore needed.

• Crontab for Linux. Crontab is pre-installed for Linux. Therefore, there is no further action needed.

4. Setting up the job queue processor

For performance reasons, a Scheduler would not be needed for machines running Enterprise Server that are configured exclusively as workflow application servers. Nevertheless, for Enterprise 10 there are only light administrative job types available. Therefore any server machine can be used, even a workflow application server. More so because those job types are typically run at night time, and therefore do not affect production at all. Nevertheless, due to its great potential, the Server Jobs concept is likely to get used more and more. This includes co-worker jobs that could take away the heavy tasks from the workflow application servers. In other terms, for Enterprise 10 it does not matter greatly which servers you pick to install Schedulers on, but for future versions it might make a big difference in performance and the deployment of additional servers becomes more important.

For continuous queue processing, the Scheduler needs to run the jobindex.php module periodically, whereby the jobindex.php runs until the Scheduler calls it again.

Example: In the following example the interval is one minute (for the Scheduler), and therefore also the duration is also one minute (for the jobindex.php). Running this URL makes the jobindex.php process jobs from the queue for one minute (60 seconds): http://127.0.0.1/Enterprise/jobindex.php?maxexectime=60&maxjobprocesses=3

Configuration steps

Some notes on the URL: Important: The double quotes ("...") around the URL are required to pass the whole URL to the curl command line tool. You can trigger remote servers too, but you might lose overview. Even worse, when the local server is down, the remote server will not be triggered anymore. A more robust and granular solution is to configure localhost (127.0.0.1) only to run on the local machine itself. Having that in place, the machine running jobindex.php is continuously watching the job queue. When it picks up a job, that job could take longer than one minute. So there is a danger that the next run of the Scheduler takes place before the first one is completed. Therefore, there is another parameter ‘maxjobprocesses’ that tells there should never be more than 3 jobs running in parallel. Doing so, the machine will not get overloaded with processes that are running jobs and swapping each other out of memory. This would be wasting processing time and leading to less throughput. Instead of picking up all jobs, you can make a specific selection: open the Server Maintenance admin page and check the Handle Jobs / Server Jobs options to specify the job types to be picked up by this machine.

5. (Optional) Setting up recurring jobs

For a whole Enterprise Server farm, there should be only one Enterprise Server instance responsible for creating recurring server jobs. This is a very light operation, so it can be performed on any machine without performance issues. (Note that this operation does not execute jobs.)

For recurring job creation, the Scheduler needs to run the jobindex.php module periodically, whereby the jobindex.php simply creates one job instance for a given recurring job type.

Enterprise 10 supports the following recurring Server Job types:

• TransferServerCleanUp. For clearing all files from the File Transfer folder.
• AutoPurgeTrashCan. For permanently deleting all files from the Trash Can.
• AutoCleanServerJobs. For deleting Enterprise Server Jobs from the Job Queue.

As an example, the following steps explain how to configure the TransferServerCleanUp job type. For other job types, please repeat all the steps but then fill in the appropriate job type name.

Here, we let the schedule run a configured URL once a day at 23:00h after working hours. Running this URL makes the jobindex.php create a Transfer Server Cleanup job in the queue:

http://127.0.0.1/Enterprise/jobindex.php?createrecurringjob=TransferServerCleanUp

Some notes on the URL: Important: The double quotes ("...") around the URL are required to pass the whole URL to the curl command line tool. You can trigger remote servers too, but you might lose overview. Even worse, when the local server is down, the remote server will not be triggered anymore. A more robust and granular solution is to configure localhost (127.0.0.1) only to run on the local machine itself. Having that in place, the job simply gets created in the queue, waiting for any Enterprise Server to pick it up. Therefore, at least one Enterprise Server needs to be configured to pick up the TransferServerCleanUp job type. Open the Enterprise Server Maintenance admin page for a configured Enterprise Server and check the Handle Jobs / Enterprise Server Jobs options to specify the job type.

6. Validating the installation

Once the Scheduler is set up, check its working by following these following steps:

Step 1. Run each of the recurring jobs manually.

Step 2. In Enterprise Server, access the Job Queue page.

The page should show the following:

• There should be one instance of each recurring job type listed.
• The Status of each job should change to Completed within one minute (after Queued Since).

Note: Refresh the page when jobs are not completed yet.

Step 3. Wait for the next day to see if the recurring tasks are automatically created at 23:00h.