An Enterprise Server Job is a task that is performed by Enterprise Server, such as clearing all objects of the Trash Can at set times or processing images during an import or export process.
This task does not necessarily have to be performed on the workflow machine: it can also be delegated to a different instance of Enterprise Server. Possible reasons for doing this include:
- Running the process is expected to take too long on the workflow machine, which would result in end users having to wait for the process to complete.
- Running the process is expected to consume too many resources (memory) on that machine, possibly causing memory swapping of other processes servicing other users, resulting in unpredictable performance.
- The workflow machine is not able to perform the task because the required tools (such as Image Magick, Tika, and so on) are not available.
Other scenarios in which Server Jobs are useful include:
- The process requires more access rights than the acting user has, and therefore there is a need to configure a user with more access rights for the job.
- The process has an administrative purpose and needs to be repeated periodically. The job can then be scheduled to run at certain times.
In all instances, Server Jobs are queued and processed asynchronously in the background. End-users therefore do not have to wait for Enterprise Server to process a job and are not affected by heavy processing or memory consumption. These are the main goals of the Server Jobs concept.
Info: This article contains references to macOS. Running Enterprise Server 10 on macOS is only supported on Enterprise Server version 10.8 or lower.
Terminology
When working with Server Jobs, the following terminology is used:
Scheduled job
A job scheduled to be repeated at set intervals. For example: clearing a temp folder every day.
User-initiated job
A job triggered by an end-user. For example: uploading an image, which results in generating a preview of that image.
Server Job Queue
Server Jobs are placed in the Server Job Queue from which they can be picked up by any Enterprise Server machine.
Server Job configuration
The settings that define a Server Job, including:
- The time interval of a recurring Server Job
- How often a recurring Server Job should run
- The number of attempts that a failed process should be retried
Background versus foreground processing
From an Enterprise Server point of view, the process of running a Server Job is done in the foreground; however, from an end-user’s point of view, the process is run in the background. This way, users can continue their work without having to wait for the process to be completed first.
Enterprise Server farm
A cluster of Enterprise Servers working together.
Server Job Processor
The Server Job Processor is the part of the system that picks up available Server Jobs from the Server Job Queue.
Co-worker machine
A Server machine running Enterprise Server that is dedicated to pick-up some or all jobs from the queue.
Note: By design, a Co-worker machine is not accessible by client applications and therefore does not act as a Workflow Server. Only under specific circumstances (for testing, demos, development, or even small offices) should a Co-worker machine be made available to client applications; doing so introduces the old issues of unpredictable performance due to heavy workload on the system.
Workflow Server
An instance of Enterprise Server set up to offer the workflow features (assigning a status, routing to users, and so on) to client applications.
Workflow Server versus Co-worker machines
There is no difference between installing Enterprise as a Workflow Server or as a Co-worker machine; each Enterprise Server system can be configured to run as a Co-worker, a Workflow Server, or both:
- When Enterprise Server is not configured to pick up any Server Jobs, it is automatically considered to be a dedicated Workflow server.
- When Enterprise Server is configured to pick up some or all Server Jobs, it is automatically considered to be a Co-worker machine.
Scheduler
A Scheduler is a continuously running engine that drives the Server Jobs mechanism for an Enterprise Server farm. It manages the following tasks:
- Continuous processing of jobs from the queue.
- Periodic creation of jobs (instances) based on recurring job configurations.
How Server Jobs are processed
Once a Server Job is created, it needs to be picked up. This can be done by each instance of Enterprise Server and is performed by the Server Job Processor: a PHP server module triggered by a scheduler.
For each process, the scheduler starts a new Job Processor. Each process runs for a very limited time (for instance a few minutes), after which the process no longer picks up new jobs from the queue and ends by itself. During the time the process runs, the scheduler creates new processes, thereby creating a pool of processes.
Idle times
When no jobs are available to be run, one process will be connected to watch for jobs arriving in the queue. This way, very low resources are used, enabling the whole server machine to become idle. When the waiting process picks up a job that just arrived from the queue, it gives way for other processes to come into action again to monitor and process the queue.
High workloads
When more jobs are available than the system can currently handle, the Server Job Processor will keep these processes on hold until the jobs that are being processed are completed.
Reattempts
In case a job fails to be processed, the system tries to understand the reason for the failure and decides if the job can be re-run. This can be repeated a set number of times after which the process is removed.
Identical jobs
In a typical setup, multiple Enterprise Servers are used. For practical reasons, the scheduling setup of recurring jobs for one Enterprise Server is usually copied to all other Enterprise Servers. As a result, multiple servers are triggering the queuing of the same Enterprise Server Job at the same time.
In an ideal setup, only one Enterprise Server is used for triggering a particular Server Job.
To prevent identical jobs from being created, Enterprise Server 10.6.0 and higher prevents the creation of a Server Job when an identical job has already been created within the last 60 seconds.
Setting up and configuring Enterprise Server Jobs
The following steps are to be performed:
1. Setting up Enterprise Server instances
2. Configuring existing Server Jobs
4. Setting up the job queue processor
5. (Optional) Setting up recurring jobs
6. Validating the installation
1. Setting up Enterprise Server instances
Each individual Enterprise Server can play several roles:
- Application Server (Workflow Server)
- File transfer Server
- Co-worker (background processor)
Some servers in an Enterprise Server farm can have multiple roles, other servers can be clustered with an Apache dispatcher, while yet other servers might be dedicated to a certain role.
Regardless of the intended server role, for each different URL through which those servers (or clusters) can be reached, follow these steps:
Step 1. In Enterprise Server, access the Enterprise Servers page.
Show me how
Step 1a. Click Integrations in the Maintenance menu or on the Home page.
Step 1b. In the Enterprise Server Jobs section, click Enterprise Servers. The Enterprise Servers page appears.
Step 2. Click New Server.
The Enterprise Server Maintenance page appears.
Step 3. In the Name box, add a descriptive name.
Step 4. From the Type list, choose Enterprise.
Step 5. In the URL field, type the URL of the server as seen from the client applications.
Example: http://127.0.0.1/Enterprise
Step 6. (Optional) In the Description field, add a description for the server.
Step 7. From the Handle Jobs list, choose All.
Step 8. Click Update.
The Enterprise Servers page appears.
2. Configuring existing Server Jobs
The following Server Jobs are available by default:
- TransferServerCleanUp. For clearing all files that are older than 24 hours from the File Transfer folder.
- AutoPurgeTrashCan. For permanently deleting all files from the Trash Can.
Note: The email address of this user is used to report the results of the Auto Purge operation by e-mail to that user. The e-mail shows how many objects there are/were in the Trash Can, how long the operation took, and any errors that occurred. For information about customizing the e-mail, see Customizing notification e-mails in Enterprise Server 10.
- AutoCleanServerJobs. For clearing up the Enterprise Server Jobs Queue. See Deleting Enterprise Server Jobs from the queue in Enterprise Server 10 for additional configuration steps.
- AutoCleanServiceLogs. (Requires Enterprise Server 10.1.7, or Enterprise Server 10.4.1 or higher) For clearing up Service logs.
- AutoCleanWebEditDir. (Requires Enterprise Server 10.4.1 or higher) For cleaning up the WebEdit folder. See also Cleaning up temp files in Enterprise.
- AutoCleanRabbitMQ. (Requires Enterprise Server 10.1.10 or any other higher version of 10.1, Enterprise Server 10.4.2 or any other higher version of 10.4, or Enterprise Server 10.5.1 or any other higher version of 10.5) For cleaning up RabbitMQ message queues. See Integrating RabbitMQ in Enterprise Server 10.
To make these operational, follow these steps:
Step 1. Access the Enterprise Server Job Config page.
Show me how
Step 1a. In Enterprise Server, click Integrations in the Maintenance menu or on the Home page.
Step 1b. In the Enterprise Server Jobs section, click Job Configurations. The Enterprise Server Job Config page appears.
Step 2. Click the name of each job and set its options:
- Attempts: Enter a number which specifies how many attempts should be made to re-try the job until running the job should be canceled.
- Username: Choose a user with system admin privileges.
- Active: Select to activate this job.
Step 3. Click Update.
3. Installing a Scheduler
A Scheduler should be set up on every Enterprise Server instance that needs to act as a Co-worker machine. This enables the Co-workers to pick up (heavy) jobs from the queue.
Step 1. Make sure that Enterprise Server is accessible from the command line, for example through the use of cURL.
Show me the instructions for installing cURL
macOS / Linux
cURL is pre-installed on these platforms; no further action is needed.
Windows
Step 1a. Download the 32-bit or 64-bit binary package from http://curl.haxx.se.
Step 1b. Extract the package into any folder, for example c:\curl.
Step 1c. Test if cURL works by starting the Command Prompt and typing the following command:
c:\Curl\curl.exe --version
The installed URL version information should be displayed.
Step 2. Use a scheduler of your choice.
- CronniX for MacOS. CronniX is a GUI for Crontab and makes its configuration a little easier. Crontab is pre-installed on Mac OS X. Download CronniX from the internet.
Note: Alternatively, you can skip the CronniX installation and use Crontab directly, but then please follow the configuration instructions written for Linux.
- Task Scheduler for Windows. Task Scheduler is pre-installed for Windows. No further action is therefore needed.
- Crontab for Linux. Crontab is pre-installed for Linux. Therefore, there is no further action needed.
4. Setting up the job queue processor
For performance reasons, a Scheduler would not be needed for machines running Enterprise Server that are configured exclusively as workflow application servers. Nevertheless, for Enterprise 10 there are only light administrative job types available. Therefore any server machine can be used, even a workflow application server. More so because those job types are typically run at night time, and therefore do not affect production at all. Nevertheless, due to its great potential, the Server Jobs concept is likely to get used more and more. This includes co-worker jobs that could take away the heavy tasks from the workflow application servers. In other terms: for Enterprise 10 it does not matter greatly which server is picked to install a Scheduler on, but for future versions it might make a big difference in performance and the deployment of additional servers becomes more important.
For continuous queue processing, the Scheduler needs to run the jobindex.php module periodically, whereby the jobindex.php runs until the Scheduler calls it again.
|
Example: In the following example the interval is one minute (for the Scheduler), and therefore also the duration is also one minute (for the jobindex.php). Running this URL makes the jobindex.php process jobs from the queue for one minute (60 seconds): http://127.0.0.1/Enterprise/jobindex.php?maxexectime=60&maxjobprocesses=3 |
Configuration steps
Steps for macOS
Step 1. Start the CronniX application.
Step 2. Choose File > Next Task.
Step 3. Access the Simple tab.
Step 4. Select the check boxes for Minute, Hour, Day of month, Month, and Day of week.
Step 5. In the Command box, type the following (make sure to add the URL in double quotes and adjust the server URL):
curl "<server url>/jobindex.php?maxexectime=60&maxjobprocesses=3"
Step 6. Click New.
Step 7. Choose File > Save.
Step 8. (Optional) In the Terminal, type the following command to see that (under the hood) the configuration is actually saved for Crontab:
crontab -e
Steps for Linux
Step 1. Edit the Crontab configuration by typing the following command:
crontab -e
Step 2. Press the [i] key to enter ‘insert’ mode and add one line below the header as follows:
|
Notes:
|
# min hour mday month wday command
* * * * * curl "http://127.0.0.1/Enterprise/jobindex.php?maxexectime=60&maxjobprocesses=3"
Step 3. Press Shift+Z twice to exit and save changes.
The saved changes are directly effective in Crontab.
Steps for Windows
Step 1. Go to Control Panel > Administrative Tools.
Step 2. Start the Task Scheduler.
Step 3. From the Action menu, choose Create Task.
Step 4. Access the General tab.
Step 5. In the Name box, type Enterprise - Job Queue Processor.
Step 6. Select the option Run whether user is logged on or not.
Step 7. Access the Triggers tab.
Step 8. Create a new trigger with the following settings:
- Begin the task: On a schedule.
- Settings: Daily, set start date & time, Recur every 1 days.
- Advanced settings:
- Repeat task every 1 minute, for a duration of 1 day, Enabled.
- Clear the option Stop all running tasks at end of repetition duration.
Step 9. Click OK.
Step 10. Access the Actions tab.
Step 11. Create a new action.
Step 12. In the Program/script box, type:
"c:\Curl\curl.exe"
Step 13. In the Add argument box, type the following (make sure to add the URL in double quotes):
"http://127.0.0.1/Enterprise/jobindex.php?maxexectime=60&maxjobprocesses=3"
Step 14. Access the Settings tab and do the following:
- Clear the option If the running task does not end when requested, force it to stop
- Set the option If the task is already running, then the following rule applies: to Run a new instance in parallel
|
Some notes on the URL:
Having that in place, the machine running jobindex.php is continuously watching the job queue. When it picks up a job, that job could take longer than one minute. So there is a danger that the next run of the Scheduler takes place before the first one is completed. Therefore, there is another parameter ‘maxjobprocesses’ that tells there should never be more than 3 jobs running in parallel. Doing so, the machine will not get overloaded with processes that are running jobs and swapping each other out of memory. This would be wasting processing time and leading to less throughput. Instead of picking up all jobs, you can make a specific selection: open the Server Maintenance admin page and check the Handle Jobs / Server Jobs options to specify the job types to be picked up by this machine. |
5. (Optional) Setting up recurring jobs
For a whole Enterprise Server farm, there should be only one Enterprise Server instance responsible for creating recurring server jobs. Calling the jobindex.php file to create recurring jobs from multiple application servers (each having an exact copy of the crontab) is NOT wanted. Enterprise Server Jobs are not designed to run in parallel; it is not wanted to have 5 identical cleanup jobs in the queue simply because 5 application servers are installed. Enterprise Server jobs therefore have to be created from one application server instance only.
Running a recurring server job is a very light operation so it can be performed on any machine without performance issues. (Note that this operation does not execute jobs.)
For recurring job creation, the Scheduler needs to run the jobindex.php module periodically, whereby the jobindex.php simply creates one job instance for a given recurring job type.
Enterprise 10 supports the following recurring Server Job types:
- TransferServerCleanUp. For clearing all files from the File Transfer folder.
- AutoPurgeTrashCan. For permanently deleting all files from the Trash Can.
- AutoCleanServerJobs. For deleting Enterprise Server Jobs from the Job Queue.
- AutoCleanWebEditDir. (Requires Enterprise Server 10.4.1 or higher) For cleaning up the WebEdit folder. Define the number of days after which the files should be removed by using the AUTOCLEAN_WEBEDITDIR_DAYS setting in the configserver.php file.
- AutoCleanRabbitMQ. (Requires Enterprise Server 10.1.10 or any other higher version of 10.1, Enterprise Server 10.4.2 or any other higher version of 10.4, or Enterprise Server 10.5.1 or any other higher version of 10.5)) For cleaning up RabbitMQ message queues. See Integrating RabbitMQ in Enterprise Server 10.
As an example, the following steps explain how to configure the TransferServerCleanUp job type. For other job types, please repeat all the steps but then fill in the appropriate job type name.
Here, we let the schedule run a configured URL once a day at 23:00h after working hours. Running this URL makes the jobindex.php create a Transfer Server Cleanup job in the queue:
http://127.0.0.1/Enterprise/jobindex.php?createrecurringjob=TransferServerCleanUp
Steps for macOS
Step 1. Start the CronniX application.
Step 2. Choose File > Next Task.
Step 3. Access the Simple tab.
Step 4. Select the check boxes for Minute, Day of month, Month and the working days of the week (Mo, Tu, We, Th, and Fr).
Step 5. In the Command box, type the following (make sure to add the URL in double quotes and to adjust the server URL):
curl "<server url>/jobindex.php?createrecurringjob=TransferServerCleanUp"
Step 6. Click New.
Step 7. Choose File > Save.
Step 8. (Optional) In the Terminal, type the following command to see that (under the hood) the configuration is actually saved for Crontab:
crontab -e
Steps for Linux
Step 1. Edit the Crontab configuration by typing the following command:
crontab -e
Step 2. Press the [i] key to enter ‘insert’ mode and add one line below the header as follows:
# min hour mday month wday command
* * * * * curl "http://127.0.0.1/Enterprise/jobindex.php?maxexectime=60&maxjobprocesses=3"
0 23 * * 1,2,3,4,5 curl "http://127.0.0.1/Enterprise/jobindex.php?createrecurringjob=TransferServerCleanUp"
|
Notes:
|
Step 3. Press Shift+Z twice to exit and save changes.
Steps for Windows
Step 1. Go to Control Panel > Administrative Tools.
Step 2. Start the Task Scheduler.
Step 3. From the Action menu, choose Create Task.
Step 4. Access the General tab.
Step 5. In the Name field, type Enterprise - Transfer Server Cleanup.
Step 6. Select the option Run whether user is logged on or not.
Step 7. Access the Triggers tab.
Step 8. Create a new trigger with the following settings:
- Begin the task: On a schedule.
- Settings: Weekly, set start date & time, Recur every 1 weeks on Monday, Tuesday, Wednesday, Thursday, Friday;.
- Advanced settings: Enabled.
Step 9. Click OK.
Step 10. Access the Actions tab.
Step 11. Create a new action.
Step 12. In the Program/script box, enter:
"c:\Curl\curl.exe"
Step 13. In the Add argument field, type the following (make sure to add the URL in double quotes and to adjust the Server URL):
"http://127.0.0.1/Enterprise/jobindex.php?createrecurringjob=TransferServerCleanUp"
Step 14. Deselect the option Stop all recurring tasks at end of repetition duration.
|
Some notes on the URL:
Having that in place, the job simply gets created in the queue, waiting for any Enterprise Server to pick it up. Therefore, at least one Enterprise Server needs to be configured to pick up the TransferServerCleanUp job type. Open the Enterprise Server Maintenance admin page for a configured Enterprise Server and check the Handle Jobs / Enterprise Server Jobs options to specify the job type. |
Step 15. (Optional, for IIS on Windows only) In the FastCGI settings of IIS, increase the value of the Activity Timeout setting. The default setting of 30 is potentially too short for letting Server Jobs to be picked up.
6. Validating the installation
Once the Scheduler is set up, check its working by following these following steps:
Step 1. Run each of the recurring jobs manually.
Steps for macOS
Step 1a. Run the CronniX application.
Step 1b. Select a configured recurring job.
Step 1c. Choose Task > Run Now.
Steps for Linux
Run any of the configured recurring jobs. For example:
curl "http://127.0.0.1/Enterprise/jobindex.php?createrecurringjob=TransferServerCleanUp"
Steps for Windows
Step 1a. Run the Task Scheduler.
Step 1b. Select a configured recurring job.
Step 1c. Click Run.
Step 2. In Enterprise Server, access the Job Queue page.
Show me how
Step 1a. In Enterprise Server, click Integrations in the Maintenance menu or on the Home page.
Step 1b. In the Enterprise Server Jobs section, click Job Queue. The Enterprise Server Job Queue page appears.
The page should show the following:
- There should be one instance of each recurring job type listed.
- The Status of each job should change to Completed within one minute (after Queued Since).
Note: Refresh the page when jobs are not completed yet.
Step 3. Wait for the next day to see if the recurring tasks are automatically created at 23:00h.
Comments
0 comments