In a typical production process, publications reach their end of life once they have been published. They are subsequently archived to make room in the system for other publications while still having access to the files for reference or re-use.
During the production process, various Dossiers will have been created as well containing specific content: articles, layouts, or reference material.
When Assets Server is integrated with Studio Server, Digital articles, layouts, and Dossiers can be archived from Studio Server to Assets Server.
Archiving can be performed when a defined number of days has passed since a particular date, or it can be triggered manually by setting a file to a dedicated Workflow Status.
This article describes the process and configuration.
Note: New archiving functionality has been progressively added to Studio Server. The steps that are described here apply to Studio Server 10.12.x. For other versions, see the following articles:
|
Scenarios
An archiving process can be run for the following objects:
- A layout including its placed objects and optionally its output files and/or the Dossier it is part of including any unplaced files.
- A Digital article including its placed objects and optionally the Dossier it is part of including any unplaced files.
- A Dossier that does not contain layouts or Digital articles (Dossiers used for storing reference material only, Dossiers containing Print articles, a Word workflow, and so on.)
You can control if archiving is performed for all these objects, some of these objects, or none of these objects.
Archiving can be performed in the following ways:
- Time based: In this scenario, archiving is performed when a defined number of days has passed since a specific date:
- The most recent modified date of an article or any of its placed files.
- The creation date of a Dossier and the most recent modified date of all files in the Dossier.
- The publication date of the Issue that a layout is part of.
- The modified date of a layout or any of its placed files.
For each type of object in a Brand (a layout, Digital article, or Dossier) one method of archiving can be defined.
How it works
The basic process of archiving files works as follows:
- A check is done to see if any files are available for archiving.
- When files are available for archiving, the archiving process is performed.
- The archived files are removed from Studio Server.
- When archiving a file failed or could not be performed, notifications are given to the user.
- Archived files can be restored at a later stage.
The main reasons why archiving could fail are:
- The Workflow Status of a file is set to 'Personal'.
- A file is open (locked) for editing.
- The Archive Status of a file is set to 'Excluded'.
- One of the files is a Layout Module or a Task.
More detailed information is given later in this article.
Requirements
It is assumed here that an integration between Studio Server and Assets Server is in place and fully working.
Configuration
Configuration steps need to be performed in Studio Server and optionally in Assets Server.
Info: Use the filter to only show information for one step. |
1. Configuring archiving settings in Studio Server
Archiving settings are configured in Studio Server for each Brand individually on the Brand Maintenance page.
The archiving options are configured in the Assets section.
Note: These options only appear when the WoodWing Assets Content Source Server plug-in is enabled, as part of integrating Studio Server and Assets Server.
Step 1. In Studio Server, access the Brand Maintenance page for which Digital articles, layouts, and/or Dossiers are to be archived. Set the fields as described in the following steps.
Note: The Production Zone field is not used for archiving and can be ignored here.
Step 2. (Only when archiving Digital articles) When images are restored as part of archived Digital articles, a default Workflow Status can be assigned to these images. Set these in the following fields:
- Default Status Restored Digital Article
- Default Status Restored Image
Notes:
|
Step 3. Determine which objects need to be archived by selecting the relevant options in the Archive Schema field:
- Digital. For archiving Digital articles.
- Dossiers. For archiving Dossiers that do not contain a Digital article or layout.
- Print. For archiving layouts.
Step 4. For each object to be archived, separate settings can be configured in the respective Archive Schema settings. For each selected Archive Schema, set its options.
- Trigger. Set as follows:
|
- Archive after (days). (Does not apply when Trigger 'Status Based' is chosen) Define the number of days after which the archiving should take place. The day that is referenced is defined by the Trigger (see previous).
Note: Carefully determine this time period: when set too short, users might still be working on files which will make the archiving process fail; setting it too long will clutter the system with unused files.
- Object type. Cannot be changed; it is automatically set:
|
- Include. Define if other available objects should also be archived:
|
- Archive Zone. Set the location in Assets Server in which to archive the files.
The structure of the location in which layouts and Digital articles and their files are archived to in Assets Server are outlined below.
Archive Schema - Digital /<base folder>/<story folder>/<version folder>/
Example 1: (Digital article is not part of a Dossier, 'Include unplaced objects in Dossier' is selected)
/Archive Zone/Amsterdam/2020-01-02 — Amsterdam/ Example 2: (Digital article is part of a Dossier, 'Include unplaced objects in Dossier' is selected)
/Archive Zone/Cities/2020-01-02 — Cities/ Example 3: (Digital article is part of a Dossier, 'Include unplaced objects in Dossier' is not selected)
/Archive Zone/Cities/2020-01-02 — Cities/ |
Archive Schema - Dossiers /<base folder>/<story folder>/<version folder>/
Example: (Dossier containing some objects but not a Digital article or layouts)
/Archive Zone/Cities/ |
Archive Schema - Print /<base folder>/<story folder>/<version folder>/
Example 1: (Layout is not part of a Dossier, 'Include unplaced objects in Dossier' is selected)
Archive Zone/Amsterdam/ Example 2: (Layout is part of a Dossier, 'Include unplaced objects in Dossier' is selected)
Archive Zone/Amsterdam/ Example 3: (Layout is part of a Dossier, 'Include unplaced objects in Dossier' is not selected)
Archive Zone/Amsterdam/ |
Setting up the structure can be done manually or automatically.
Manual setup
In this setup, a folder is manually created and linked to in the Studio Server settings.
Step 4a. Using Assets, create a folder on Assets Server and assign Permissions to it.
Note: It is recommended to not allow users to edit archived files.
Step 4b. Add the path to the Archive Zone field.
Example: /Archive Zone/My Brand/My publication date
Automatic setup
In this setup, a folder structure is constructed in the Brand settings on Studio Server. When archiving is performed for the first time for that Brand, the folder is automatically created in that location in Assets Server (when it does not exist yet). User permissions are set once the folder exists.
Step 4a. Construct the path by using the following variables.
Notes:
Example:foo/${issue}/bar results in foo/bar and not foo//bar if the issue is unknown. |
Wildcard | Description |
---|---|
${brand} | Adds the name of the Brand to the folder name. |
${category} |
Adds the name of the Category to the folder name. |
${issue} |
Adds the name of the Issue to the folder name. Notes:
|
${publicationdate:Y-m-d} |
Adds the year, month, and day of the publication date of the Issue to the folder name. Notes:
|
${archived:Y-m} |
Adds the year and month to the folder name of the moment when the archiving takes place. Example: (When date is 1 January 2020): ${archived:Y-m} becomes 2020-01. |
${date:Y-m} |
Adds the year and month to the folder name of the moment when the archiving takes place. Example: (When date is 1 January 2020): ${date:Y-m} becomes 2020-01. |
${created:Y-m} |
Adds the year and month of the moment when the layout, Digital article, or Dossier was created to the folder name. Example: (When date is 1 January 2020): ${created:Y-m} becomes 2020-01. |
${modified:Y-m} |
Adds the year and month to the folder name of the moment when the layout, Digital article, or Dossier was modified. Example: (When date is 1 January 2020): ${modified:Y-m} becomes 2020-01. |
${C_<name>:Y-m} |
Adds the year and month to the folder name of a custom workflow property (specified by C_<name>) of which the type is set to 'Date' or 'Datetime'. Example: (When custom property is C_MY_PROPERTY and date is 1 January 2020): ${C_MY_PROPERTY:Y-m} becomes 2020-01. |
${C_<name>} | Adds the value of a custom workflow object property (specified by C_<name>) to the folder name. Replace C_<name> with your custom property name. |
Example: The following definition: /Archive Zone/${brand}/${category}/${created:Y}/${created:m} results in the following path (when the Brand is 'WW News', the Category is 'Sport', and the current date is 1 January 2020): /Archive Zone/WW News/Sport/2020/01 |
Step 4b. Once a first archiving action is performed for the Brand and the defined folder exists in Assets Server, assign Permissions to the folder.
Step 5. Click Update to save the changes.
2. Setting up Studio Server Jobs
In this step, scheduled tasks are set up in Studio Server which perform the archiving process.
Note: It is assumed here that the Studio Server Jobs feature is set up and fully functioning. Fore more information, see Working with Studio Server Jobs in Studio Server.
This process is performed by the following Studio Server Jobs:
- SelectForArchive. Determines if any objects (a layout, Digital article, or Dossier) are available for archiving. When found, it creates a SendToArchive job for that object.
- SendToArchive. Performs the actual archiving for that object.
These jobs need to be configured and a recurring job needs to be set up.
Step 1. On the Studio Server Job Config page click on the Server Job names SelectForArchive and SendToArchive to access their settings page and in the User name field set a user who has system admin privileges. Click Update when done.
Step 2. Set up a recurring job for the SelectForArchive job.
Example: A crontab configuration for running the job every hour: 0 * * * * curl "http://myserver.com/StudioServer/jobindex.php?createrecurringjob=SelectForArchive&serverplugin=Elvis" |
Note: When Status based archiving is used, it is recommended to disable archiving during production hours and to let the recurring SelectForArchive jobs run during non-production hours only. This avoids SendToArchive jobs from being created in the job queue, thereby essentially disabling the archive feature during that period. This prevents scenarios where users edit files while other users accidentally indirectly send these files to the archive through status changes of related objects. |
3. (Optional, for Digital articles only) Configuring versioning settings in Studio Server
Info: This step only has to be performed when archiving Digital articles.
When making use of this archiving method, the Create Permanent Version option should be disabled for images, videos, and audio files.
This can be done in the Workflow Status settings for these objects.
4. Testing the implementation
To test if the configuration has been set up successfully, do the following:
- For Issue based archiving and for Continuous archiving: A simple demo tool is available for testing the archiving feature and thereby better understanding the system behavior. It is explained in the following file as part of the Enterprise Server package:
StudioServer/server/plugins/Elvis/demo/sendtoarchive/README.md
Note: Do not use the tool in a production environment.
Use the criteria described in the archiving workflow section in this article for setting up your testing scenarios.
- For Status based archiving: Assign a Digital article, Dossier, and/or layout to a Workflow Status for which 'Send to archive' has been enabled. Wait for the Server Job to be processed and verify that the files have been archived.
5. Optional steps
After setting up the integration itself, the following optional steps may be required to perform:
1. Adding the 'Archive Status' property to the clients
In certain scenarios, a Brand administrator may want to manually modify the 'Archive Status' property of a file.
Examples:
|
To make this possible, the 'Archive Status' property needs to be made visible in the client applications. For information about how this is done, see Adding a property to a dialog box in Studio Server.
2. (For layouts only) Configuring how images should be restored
When an archived layout is restored to Studio Server from Assets Server, any images that are placed can be restored in one of the following ways:
- As a copy in Studio Server that is not linked to the file in Assets Server.
- As a Production shadow object in Studio Server that is linked to a copy of the original file in Assets Server.
For information about configuring this, see Controlling how files are stored in the Assets Server - Studio Server integration.
3. (For Digital articles only) Making it possible to preview archived Digital articles in Assets
Info: This step only has to be performed when archiving Digital articles.
Digital articles make use of various resources such as Component Sets, styles, custom swatches, and so on. These resources are stored in a central location in the cloud in a system named WoodWing Cloud Services. To make it possible to preview a Digital article in Assets, Assets Server and Studio Server need to be connected to WoodWing Cloud Services so that these resources can be shared.
This is done through a so-called tenant registration. For Assets Server this registration takes place in the Management Console.
Perform this process by following the steps in Connecting Studio Servers and Assets Server to WoodWing Cloud Services.
The archiving workflow
The archiving process takes place in two parts:
- At a scheduled time, a check is done to see if any objects (a layout, Digital article, or Dossier) are available for archiving. Such objects are referred to as 'candidate objects'.
- For each found candidate object, a Server Job is created. When this job is run, the following processes are performed:
- The objects to archive are determined and verified. These are referred to as 'invoked objects' and include:
- The candidate object itself
- Any objects placed on the candidate
- Any other files to archive along with the candidate (such as unplaced files that are also in the Dossier)
- The archiving process is performed.
1. Determining candidate files
Specific rules apply for determining if a layout, Digital article, or Dossier is scheduled for archiving.
Process for Digital articles
A Digital article is included for archiving when all of the following conditions are met:
- The Digital article is part of a Brand for which the Archive Schema option 'Digital' is selected.
- The 'Archive Status' property of the Digital article is empty.
- When Trigger 'Continuous' is selected:
- The difference between the current date and the modification date of the Digital article and all of the placed files is higher than the number set in the 'Archive after (days)' option.
- When Trigger 'Status Based' is selected:
- The Digital article is assigned to a Workflow Status for which the option 'Send to Archive' is enabled.
When a candidate Digital article is found, the following actions take place:
- The 'Archive Status' property of the Digital articles is set to 'Candidate'.
- The 'Archive Status Modified' property of the Digital articles is set to the current date and time.
- Studio Server Jobs are created for archiving the Digital articles (one Job for each candidate article).
- The Studio Server Job that was used for determining the candidate Digital articles is automatically removed from the job queue.
Process for Dossiers without layouts or Digital articles
A Dossier that does not contain layouts or Digital articles is included for archiving when all of the following conditions are met:
- The Dossier is part of a Brand for which the Archive Schema option 'Dossiers' is selected.
- The 'Archive Status' property of the Dossier is empty.
- When Trigger 'Continuous' is selected:
- The difference between the current date and the creation date of the Dossier as well as the modified date of all files in the Dossier is higher than the value set in the 'Archive after (days)' option for the Brand.
- When Trigger 'Status Based' is selected:
- The Dossier is assigned to a Workflow Status for which the option 'Send to Archive' is enabled.
When a candidate Dossier is found, the following actions take place:
- The 'Archive Status' property of the Dossiers is set to 'Candidate'.
- The 'Archive Status Modified' property of the Dossiers is set to the current date and time.
- Studio Server Jobs are created for archiving the Dossiers (one Job for each candidate Dossier).
- The Studio Server Job that was used for determining the candidate Dossiers is automatically removed from the job queue.
Process for layouts
A layout is included for archiving when all of the following conditions are met:
- The layout is assigned to a Brand for which the Archive Schema option 'Print' is selected.
- The 'Archive Status' property of the layout is empty.
- When Trigger 'Issue based' is selected:
- The layout is assigned to an Issue for which a publication date is set.
- The difference between the current date and the publication date is higher than the number set in the 'Archive after (days)' option.
- When Trigger 'Continuous' is selected:
- The difference between the current date and the modification date of the layout and all of the placed files is higher than the number set in the 'Archive after (days)' option.
- When Trigger 'Status Based' is selected:
- The layout is assigned to a Workflow Status for which the option 'Send to Archive' is enabled.
When a candidate layout is found, the following takes place:
- The 'Archive Status' property of the layout is set to 'Candidate'.
- The 'Archive Status Modified' property of the layout is set to the current date and time.
- Studio Server Jobs are created for archiving the layouts (one job for each layout).
- The Studio Server Job that was used for determining the candidate layouts is automatically removed from the job queue.
Note: As mentioned earlier in this article, even though Status based archiving is a manual way of archiving files, the actual archiving process is a scheduled process and is therefore determined by how often this process is run. Technically it is therefore also time based. Depending on how much time remains between assigning the file to the archive status and the moment the archiving process is run, users may or may not experience a delay in seeing the files being archived (this can also be advantageous because it can allow users who change their mind to remove the status again when sufficient time is available).
2. Performing the archiving process
When candidate objects are found and all validation rules are met, archiving takes place.
Specific steps apply for each archived object.
Process for Digital articles
- The Digital article and all its placed objects are archived. These files are re-linked to the article to make it possible to restore the article when needed.
- When the 'Include unplaced objects in Dossier' option is selected and the Digital article is part of a Dossier, all other files in the Dossier are also archived.
- The name of the folder in which the files are archived is the name of the Digital article, or (when the 'Include unplaced objects in Dossier' option is selected and the Digital article is part of a Dossier) the name of the Dossier.
- Any shadow objects that were stored in the Production Zone of Assets Server are moved to the Archive Zone in Assets Server; when these files are still used elsewhere, they are copied instead of moved. These images are relinked in the Digital article.
- (For Assets Server 6.50 and higher) The names of the default components that the Digital article contains are stored in a metadata property named 'Article Components'. This property can be used for finding Digital articles containing a specific component, for example by adding the property to the Filter in Assets.
Note: Names of custom properties are not stored.
- The archived files are removed from Studio Server.
Note: A Dossier may be left in place if it still contains files that have not been archived.
- For Digital articles that could not be archived, the Archive Status property is set to 'Failed' and messages are added to the files to inform the user.
A Digital article is not archived when:
- The Archive Status of the Digital article or one of its placed files is set to 'Excluded' or 'Failed'.
- The Workflow Status of the Digital article is set to 'Personal'.
- The Digital article or any of its placed images is open (locked) for editing.
- The Digital article is part of a Task.
- The Digital article is part of a Dossier template.
- The Digital article is part of multiple Dossiers.
- The Digital article is part of a single Dossier but:
- The Dossier is set to the 'Personal' status.
- The Dossier contains a Task.
- When 'Include unplaced objects in Dossier' is selected:
- The Archive Status of the Dossier or one of its objects is set to 'Excluded' or 'Failed'.
- The Workflow Status of the Dossier or any of its objects is set to 'Personal'.
- An object in the Dossier is open (locked) for editing.
- An object in the Dossier is a Layout Module or a Task.
- The difference between the current date and the modification date of at least one file in the Dossier is lower than the value set in the 'Archive after (days)' option for the Brand.
Process for Dossiers without layouts or Digital articles
- The Dossier and all its objects are archived.
- The archived files are removed from Studio Server.
Note: Files that are still used in other Dossiers are not removed.
A Dossiers is not archived when:
- The Archive Status of the Dossier or one of its objects is set to 'Excluded' or 'Failed'.
- The Workflow Status of the Dossier or any of its objects is set to 'Personal'.
- An object in the Dossier is open (locked) for editing.
- An object in the Dossier is a Layout Module or a Task.
- The difference between the current date and the creation date of the Dossier as well as the modified date of all files in the Dossier is lower than the value set in the 'Archive after (days)' option for the Brand.
- For Dossiers that could not be archived, the Archive Status property is set to 'Failed' and messages are added to the files to inform the user.
Process for layouts
- The layout and all its placed objects (including those on the pasteboard) are archived. These files are re-linked to the layout to make it possible to restore the layout when needed.
- When the 'PDFs and EPS files' option is selected these output files are also archived.
Note that an output file is created for each page and for each Edition. Many output files can therefore exist for each layout. Each output file is included in the layout folder and will have the following file name structure:
layout name — Edition name— page number.file extension
Example: Amsterdam — North — 1.pdf
- When the 'Include unplaced objects in Dossier' option is selected and the layout is part of a Dossier, all other files in the Dossier are also archived.
- When a Word or Excel file is placed on the layout, an InDesign preference controls whether the original file is placed or the placement is a link to the original file (see Preferences > File Handling > Create Links When Placing Text and Spreadsheet Files).
Depending on how this preference is set, Word and Excel files are treated as follows:
- Option is disabled: The Word or Excel file is automatically converted to an InCopy file by Studio when it is placed. During archiving, this InCopy file is archived.
- Option is enabled. A link is created to the original Word or Excel file. During archiving, this original file is archived.
- The archived files are removed from Studio Server.
Note: A Dossier may be left in place if it still contains files that have not been archived.
- The name of the folder in which the files are archived is the name of the layout.
- For files that could not be archived, the Archive Status property is set to 'Failed' and messages are added to the files to inform the user.
A layout is not archived when:
- The Archive Status of the layout or one of its placed files is set to 'Excluded' or 'Failed'.
- The Workflow Status of the layout is set to 'Personal'.
- The layout or one of its placed files is open (locked) for editing.
- The layout is part of an InDesign Server Job.
- The layout contains a Layout Module.
- The layout is part of a Task.
- The layout is part of a Dossier template.
- When 'Include unplaced objects in Dossier' is selected:
- The Archive Status of the Dossier or one of its objects is set to 'Excluded'.
- The Workflow Status of the Dossier or any of its objects is set to 'Personal'.
- An object in the Dossier is open (locked) for editing.
- An object in the Dossier is a Layout Module or a Task.
- When Trigger 'Issue based' is selected:
- The layout is assigned to an Issue for which no publication date is set.
- The difference between the current date and the publication date is lower than the number set in the 'Archive after (days)' option.
- When Trigger 'Continuous' is selected:
- The difference between the current date and the layout modification date is lower than the number set in the 'Archive after (days)' option.
Advanced scenarios
Take note of the following advanced scenarios:
# | Scenario | Result |
---|---|---|
1 | A Dossier contains multiple Digital articles. | Each Digital article is archived separately; placed objects and optionally the other objects in the Dossier are also archived separately (creating multiple instances in Assets Server if these files are shared between the Digital articles). |
2 | A Dossier contains a Digital article and a layout. | The Digital article is archived first, the layout is archived next. Placed objects and optionally the other objects in the Dossier are also archived separately (creating multiple instances in Assets Server if these files are shared between the Digital articles or layouts). |
3 | A Digital article is part of multiple Dossiers. | The archiving process fails. |
4 | A layout is part of multiple Dossiers. | The layout is archived together with all of its placed and optionally unplaced files from all Dossiers. |
5 | A layout or Digital article is part of a Dossier and a Dossier template. | The layout or Digital article is archived and removed from the Dossier but remains in the Dossier template. When 'Include unplaced objects in Dossier' is selected, the Dossier is removed. |
6 | Multiple Server jobs are executed in parallel containing the same invoked objects. | Only one Server job is run, the others are put on hold until it is completed. |
Notifications when archiving fails
When for some reason the archiving process cannot be performed, users are notified in the following ways:
- A Flag icon is shown to the user for the candidate object and all invoked objects:
- In Studio for InDesign in the Flag (Icon) column of the Studio panel when viewing files in List View mode.
- In Studio in the Flag column of the search results panel when viewing files in List View mode.
When hovering the mouse pointer over the icon, a tooltip with information about why the object could not be archived is shown.
- For Dossiers, a Dossier note is added which can be viewed in Studio.
Preventing files from being archived
Preventing files from being archived can be done in various ways. Multiple scenarios exist when files are excluded (see above), but to quickly prevent candidate objects from being archived, use one of the following methods:
- Preventing all objects of a certain type from being archived is done by clearing its option in the Archive Schema list on the Brand Maintenance page:
- Digital. Disables the archiving process for Digital articles.
- Dossiers. Disables the archiving process for Dossiers that do not contain a layout or Digital article.
- Print. Disables the archiving process for layouts.
- Preventing a candidate object from being archived is done by setting its 'Archive Status' property to 'Excluded'. When the candidate object is part of a Dossier and 'Include Unplaced objects in Dossier' is selected, none of the files in the Dossier are archived.
- Preventing layouts from being archived when Trigger 'Issue based' is selected on the Brand Maintenance page is done by clearing the publication date for an Issue.
- Preventing layouts, Digital articles, and Dossiers from being archived when Trigger 'Status based' is selected on the Brand Maintenance page is done by clearing the option 'Send to Archive' for all Workflow Statuses in which this is set.
Preventing the archiving from failing
The exact conditions in which archiving fails are described above, but in general follow these guidelines:
- When Tasks are used, remove them from the workflow as soon as they are completed.
- When Layout Modules are used, consider disabling the archive schema 'Print'.
- Do not enable archive schemas for objects that are not used (such as 'Digital' when the Brand is used for print only).
- Do not keep files checked out for editing.
- Do not use the Workflow Status 'Personal' for files that can be picked up for archiving (such as files in a Dossier).
- Set the 'Archive after (days)' option to a suitable period of time (not too short and not too long).
- Perform Status Based archiving during non-production hours only.
Locating files that have failed to archive
It is important to be aware of files that have failed to archive and to rectify them. For this purpose, a Custom Search can be set up in Studio to locate these files on a regular basis.
For more information, see Locating files in Studio that failed to archive.
Resolving failed archiving
When the archiving process fails, the objects need to be manually corrected to have them picked up as a candidate again. Do the following:
- Clear the Archive Status field for that object (it will be set to 'Failed').
- Remove an object from a Dossier when the Workflow Status of that object is set to 'Personal'.
Restoring archived files
Information about restoring archived files back to Studio Server can be found in the following articles:
- Digital articles: By opening them in Assets.
- Dossiers: By manually creating a Dossier in Studio and dragging the files from Assets into it.
- Layouts: By opening them in Assets.
- Unplaced files: By dragging them from Assets into a Dossier in Studio.
Comment
Do you have corrections or additional information about this article? Leave a comment! Do you have a question about what is described in this article? Please contact Support.
0 comments
Please sign in to leave a comment.