8th Hussars Regimental Museum Digital Preservation Plan

Option 1 – Multiple Backups. Offsite CANB archive. Preservation Copies for other content.

Clean up the physical media – move existing CDs to the office machine hard drive, then to two external hard drives.^{Footnote 1}

Finish Digitising from analog media (cassette, film, etc… reel to reel) as time and budget permits.  Make copies to the office machine hard drive, and two external hard drives.

For digitized images, audio, and film:  Create a naming system that ensures each resource has a unique filename. Record that unique filename in the archival database under the record for that resource. This allows digitized resources to be searched using the database.

Keep all working copies of all digital assets (all groups) on the internal hard drives of the office computers. These computers are password protected at the operating system level (i.e. one must provide a password to use the machine).

Make regular backups of the relevant directories for all media groups to two external hard drives.

Refresh media by replacing the external hard drive every five years. The office PC (and thus its internal drive) will likely need replacement at a similar interval, as the weight of newer software typically bogs down machines older than this.

Migrate to new file and software formats as required.

The management of administrative documents is not part of this preservation plan, but these can also be copied to external hard drive(s) to ensure backup versions exist.

At any time that migration takes place, ensure the migrated files are backed up to the external drives.

Keep at least one external hard drive offsite.

Option 1 Pros

• Addresses immediate need to move digital assets from non-archival CDs.
• Simplest solution – can be implemented without significant training.
• Affordable – requires only the purchase of a second external hard drive.
• Provides better archival protection than status quo.

Option 1 Cons

• No Fixity checking (no process to determine if a file has changed from its original state).
• Keeping all content on hard drives does not observe the 3-2-1 rule^{Footnote 2}. 
• No audit trail is archival content is changed.
• The archive is susceptible to viruses being transferred to it.

Option 2 – Same as option 1, but include use of a checksum generator.

Checksum generators use a function to produce a unique value based on the contents of a file. If the file’s contents change even by a single bit, the same checksum will not be produced the next time the generator is run, indicating that the file is not the same; i.e. it’s “fixity” can be assessed.

Various checksum generators are considered; The first reviewed is the Duke Data Accessioner, a free application that ingests content by making a copy of a file (or several files) into an archive, generating basic preservation metadata (such as: who preserved the content, when, short description of the content), and then generating checksum information. Versions of this software can be run on several common operating systems and it is accompanied by an operator’s manual.

A second application, the MD5 Checker, is also free. It creates checksum information only – no other preservation metadata. However, it is easier to use than the Duke Data Accessioner, and was sufficiently intuitive that no consultation of the manual (which is available online) was required.

Option 2 Pros:

• Fixity of files can be easily assessed.
• Using a checksum is in keeping with CANB (an indeed all) archival standards.

Option 2 Cons:

• Some amount of training may be required.
• The Checksum generation represents an extra step in the archival process.

Option 3 – Same as option 2, but include also a quarantine machine.

Quarantine machines are often added between machines that hold working copies of content to be archived, and the archive itself. Content to be archived is transferred to the quarantine machine and held there for a “quarantine” period to ensure files are not infected before the files are then transferred to the archive.

Quarantine machines are generally used where content is brought to an archive (from an external source) for archiving. While this is not the case for the 8^th Hussars (all content destined for an archive is generated internally), there remains the possibility of infection through any machine connected to the internet. The 8^th Hussars currently has two computers; the first manages only archival records and this one is not connected to any network. The second machine is used for all other activity including email and other internet activities.

Adding a quarantine machine between the archival records machine and the hard drive that will be used to archive digital content will add little value, as the archival records machine is already isolated. In fact, a quarantine machine may serve as a vector for transferring viruses from the main office machine to the archival records machine. Instead, it would make more sense to dedicate an external hard drive exclusively to the archival records machine. This external hard drive would be plugged into the archival records machine only – never to any other computer.

A quarantine machine between the main office machine and the digital records archive would offer some protection. Anti-virus software might be run on the working machine prior to transferring files to the quarantine machine.  Files would then be transferred to the quarantine machine using a temporarily connected “crossover” cable, and would sit on the quarantine machine for the quarantine period. If, by the end of that period no viruses are detected, archive copies would then be transferred from the quarantine machine to an external drive. If a virus is detected during the quarantine period, both machines would be scanned and cleaned of the virus, and no files would be transferred for that timeframe.

Pros and cons will be considered for a quarantine machine in this configuration only. I.e. a configuration in the following form:

• Office machine → quarantine machine → archive external drive 1
• Archive records machine → archive external drive 2

Option 3 Pros

• A quarantine machine provides a degree of protection against viruses that could not be detected by the working machine’s anti-virus software (possibly because the virus is too new at the time of infection).

Option 3 Cons

• An additional copy of the files to a quarantine machine constitutes an additional link in the “chain of authenticity”, meaning there is an additional intermediate copy of a file between the original and the archived version.
• A dedicated quarantine machine and the additional anti-virus software cost money.
• The quarantine process doubles the activity required to archive files.
• The greater the degree of virus protection desired, the longer the necessary quarantine period (so as to give anti-virus software makers time to identify and address new threats). This increases the risk of losing a resource that has not yet been archived (either because it was accidentally modified, deleted or overwritten).

Option 4 – Create an internal archive using the OAIS model.

In this option, the simplest form of software necessary to create an OAIS compliant model is considered. Archivematica was reviewed for this purpose, and found to be substantially too onerous (both in installation costs, and operational complexity) to be practical for any museum, save possibly the largest. No other OAIS-compliant applications were found to be as affordable as Archivematica, nor substantially simpler to operate.

Pros and cons for this option are not considered, as the option is not considered realistic for a small museum.

Option 5 -  Use an external archive

If an external organisation could manage an OAIS-compliant archive for use by the 8^th Hussars museum and similar organisations, the responsibility of refreshing and migrating could be completely offloaded. Moreover, the work involved for the museum in accessioning and accessing content might be reduced. No known service exists at present (at least none within the budget constraints of the museum). Should such a service be identified, a cost/benefit analysis would have to be performed at that time. Some major concerns that would have to be addressed include:

• The archive’s ability to manage disparate forms of data (such as the Filemaker Pro records).
• The ease with which content can be searched and retrieved.
• Bandwidth / time considerations for uploading and downloading content.
• Sustainable finances for the museum (where fees for using the archive exist), sustainable finances for the archive, and assurances (legal obligations) of the archive to maintain the museum’s content.

Pros and cons for this option are not considered at this time, as no such external archive has been identified.

Risks Inherent in the Scheduled Backup Software and the Creation of Preservation Copies

To address the risk of propagating overwrites, version control should be used where possible – when saving a modified file (be it an image file, audio, text, etc…), it should always be saved to a new name. If this is not done, the file that overwrote can still be renamed before the weekly backup is run. The target drive will then keep both versions (the overwritten version being moved to its “archive” folder).

Likewise, to address the risk of unwanted changes to databases, the copy of the database residing on the backup should be moved every week to the target drive’s archive folder (i.e. the folder that holds files which are no longer found on the source drive), and this file should be renamed to incorporate the date on which it was backed up (using the yyyymmdd format). There is ample space on the 2 TB drives to make backups of collections management or archival database files every week of the month (4-5 copies), and every month of the year (12 copies). Weekly copies should be overwritten in subsequent months, so that no more than 5 weekly copies exist. Likewise, no more than 12 monthly copies should exist.

There is also the risk that an overwrite or an erroneous change to a database is not noticed in time to correct the mistake (and the error is propagated to the target drive). To mitigate this problem, once a year (starting immediately after the first backup) the root of the backup (and all its files and subdirectories) on the target drive should be copied to a new directory on the target drive using MD5 checksums to ensure file fixity (note that Bvckup 2 uses MD5 checksums, but these are stored in a proprietary format – making the information unusable by any other application – for this reason, copies to the new directory should be made with MD5 Checker). The new directory name should include the date that the copy was made (in yyyymmdd format), and using the MD5 checker to produce this copy will generate MD5 checksums for all files. This checksum information should also be stored in the root of the new directory.

Despite its lack of other preservation metadata, this copy will be referred to as the “preservation” copy, and it should never be deleted or modified.

Managing Preservation Copies

For most small to medium-sized museums, a 2 TB external drive offers sufficient space for more than five years’ worth of preservation copies. At the end of the 4th year, the directories on the external hard drive should appear as follows:

• < Preservation copy backed up 20150430>
• < Preservation copy backed up 20160430>
• < Preservation copy backed up 20170430>
• < Preservation copy backed up 20180430>
• <Current Backup>

A second external hard drive should also be used to maintain copies of the above folders, and such copies can be made once a month (starting immediately after the first backup). Note that the contents of folders entitled “Preservation copy backed up …” do not change, and for that reason, these only need to be copied to the second external hard drive once. Contents of the “Current Backup” folder change weekly, and this will need to be overwritten on the second external drive each time the two are synchronised.

The first of the two external hard drives can be stored in the office, and the second should be kept offsite – at an executive committee member’s home for instance.

Backing up Content on the Archival Computer

All archive records residing on the 8^th Hussars museum archival computer, are now uploaded to the CANB database, meaning two copies of this database are already in existence – an arrangement should be made with CANB to ensure content held by them remains current. A third external hard drive (which can be stored in the office, and which should never to be plugged into any other computer) can be used to create a third copy of this archive. This procedure is a simple copy of the archives database folder onto the third external hard drive once a week. The number of weekly backup folders should be kept to 5 (one for every week of the month) with the each weekly folder being overwritten in successive months.

On the first weekly backup of each month, a copy of that backup should also be made to the same drive under a folder bearing the name of that month. The number of monthly backup folders should be kept to 12 (one for each month).

Once a year (starting with the first backup), a copy of the most recent weekly backup should also be made using MD5 checker, and checksum information for all files in it should be added to that folder. Again, despite the lack of other preservation metadata, these annual backups are referred to as “preservation copies of the archival database. The folder name should indicate that it is a preservation copy of the Archives database, and it should include the year and month in which it was created.

If backups were first started in June 2015, then after two years under such a protocol, the root directory of this external drive will appear as below:

• < 001-Weekly Archives Database backup>
• < 002-Weekly Archives Database backup>
• < 003-Weekly Archives Database backup>
• < 004-Weekly Archives Database backup>
• < 005-Weekly Archives Database backup>
• < 01-Jan Monthly Archives Database backup>
• < 02-Feb Monthly Archives Database backup>
• < 03-Mar Monthly Archives Database backup>
• < 04-Apr Monthly Archives Database backup>
• < 05-May Monthly Archives Database backup>
• < 06-Jun Monthly Archives Database backup>
• < 07-Jul Monthly Archives Database backup>
• < 08-Aug Monthly Archives Database backup>
• < 09-Sep Monthly Archives Database backup>
• < 10-Oct Monthly Archives Database backup>
• < 11-Nov Monthly Archives Database backup>
• < 12-Dec Monthly Archives Database backup>
• <201506 Archives Preservation Copy>
• <201606 Archives Preservation Copy>
• <201706 Archives Preservation Copy>

Testing Fixity

Once a year, all checksum information should be tested to ensure files in preservation folders have not changed (i.e. testing for fixity). This can be done with the MD5 Checker software. The protocol for dealing with a changed file will depend on what is discovered. If for instance, a number of files on a drive are found to be unreadable, it indicates a problem with the drive itself, and the drive should be replaced. If however, a single file has a changed checksum and is still readable, it may be nothing more than an inadvertent file write, and the original file might be recoverable from an alternate preservation copy.

Refreshing Media

When the external hard drives are first purchased, labels should be added to them, indicating what the drive contains (i.e. Preservation Copies 1 Working Machine / Preservation Copies 2 Working Machine / Preservation Copies Archives Machine)  as well as a notice to “Replace on MMM YYYY”, where “MMM YYYY” is the date five years from the drive’s date of purchase. The computer’s internal drives should also be replaced, but this is likely to happen with regular renewal of office equipment, and is less crucial than refreshing drives holding the preservation copies.

When preservation drives are replaced, a straight copy of everything from the older drives to the newer ones can be made by connecting both to USB ports on the same machine and performing a straight copy (drag of folders) from one to the other. The MD5 Checker should be used prior to the copy to ensure that existing checksums match current checksum to establish “fixity” (i.e. that the files have not changed in any way).

The amount of space used in the previous five year’s backups should be reviewed and used to extrapolate the need for additional space in the upcoming five years. The decreasing cost and increasing performance of storage space is likely to keep pace with any small museum’s archival needs, but should space become a problem, preservation copies from every other year (starting with the earliest years first) can be pruned during the refreshing process to reduce space requirements on the new drives.

File Formats and Data Migration

A list of software applications and their associated file formats should be maintained.  If a new application is being considered for the replacement of an existing system, or if a newer file format is being considered, a file migration plan will need to be developed at that time.

Also, when planning a migration to new file formats or software, the use of formats conducive to long-term preservation should been kept in mind.

Administrative documents (asset group 2) is, strictly speaking, generally not treated in a preservation plan, but it is included here, as the process for preservation of all content is sufficiently simple to incorporate all groups without additional effort. These documents are unlikely to require a migration plan, as newer software (word processing, spreadsheets, etc…) are sufficiently backwards-compatible to ensure the 7-year retention requirement for these documents is met.

To do immediately:

• Purchase, or ensure the museum has in its possession three new external USB Hard Drives, each with at least 2 Terabyte (TB) of disk space ($300 total).  Label the drives as:
  • Preservation Copies 1 Working Machine – Replace on MMM YYYY
  • Preservation Copies 2 Working Machine – Replace on MMM YYYY
  • Preservation Copies Archives – Replace on MMM YYYY

  …where MMM YYYY is the date five years from the purchase date of these drives.

• Establish a protocol with CANB to ensure the content on the archives machine and the CANB database remain synchronised on a regular basis.
• Download a copy of Bvckup 2 backup software, and install this on the office (working) machine. Plug into this machine, the first external hard drive (labelled “Preservation copies 1 Working Machine”), and schedule a weekly backup to run when the machine is on, but not likely to be used. In this scheduled backup:
  • Identify as source directories, all those containing content in asset groups 1-3 & 5-7.
  • Ensure deleted files on the source drive are archived on the target drive.
  • Name the destination folder on the target drive as “Current Backup”
• Download and install a copy of MD5 Checker, on the office machine.
• Copy all contents of existing CDs (and in particular, contents of all non-archival CDs) to the hard drives on the office computers (archival content being copied to the archival computer, and all other content being copied to the office working computer).
• Finish Digitising from analog media (cassette, film, etc… reel to reel) as time and budget permit. Content should be stored to the working drive of the appropriate (office / archival) computer.
• After the first scheduled backup (the first backup can be run early if desired) rename the “Current Backup” folder to “Preservation copy backed up yyyymmdd” where yyyymmdd is the date on which the backup was run. Open MD5 Checker, and run it on all files under the new “Preservation copy” folder. Store the MD5 checksum information in that folder.
• Plug in the second external hard drive and copy to it the contents of the first external hard drive.

Perform the following as time permits:

• For digitized images, audio, and film: Create a naming system that ensures each resource has a unique filename.  An ideal filename convention is one that contains the record ID for any database with which it is associated (an accession code for instance). Likewise for digitized media that is associated with either the archival or collections management databases, assign usage of an unused field in those databases to media filenames, and record those filenames in that field. This allows digitized content to be quickly located through database searches.

To be performed routinely:

• Maintain existing password protection on both of the museum’s computers. Doing this at the operating system level (i.e. requiring a password to log onto the machine) is sufficient for a small museum.
• Store the first and third external hard drives (“Preservation Copies 1 Working Machine” & “Preservation Copies 2 Working Machine”) in the museum office. Keep the second external hard drive (“Preservation Copies Archives Machine”) offsite - at an executive committee member’s home for instance.
• Use version control (i.e. name files according to the sequence in which they were edited either with a version number or a modification date) on all documents that will be edited. Rename any file that is mistakenly overwritten so that the original file will be archived on the backup drives.
• Migrate to new file and software formats as required. See “File Formats and Data Migration” for details.

To be performed on a weekly basis:

• Review and verify that all asset groups are within the scope of the weekly backup (i.e. that all the relevant directories are listed in the weekly backup). Add any that are not.
• Make a copy of the collections management database on the working (office) machine to the “archives” subfolder of the “Current Backup” folder on the external hard drive labelled “Preservation Copies 1 Working Machine”, as per the “Risks Inherent in the Scheduled Backup Software and the Creation of Preservation Copies” section of this document.
• Run (or allow to automatically run) the weekly backups of the relevant directories of all asset groups on the working computer to the external hard drive labelled “Preservation Copies 1 Working Machine”, as per the “Using a Scheduled Backup” section in this document.
• Manually create a copy of the archives database to the (third) external hard labelled “Preservation Copies Archives”, as per the “Backing up Content on the Archival Computer” section in this document.

To be performed on a monthly basis:

• Make a monthly copy of the most recent weekly backup of the collections management database which is located in the “archives” subfolder of the “Current Backup” folder on the external hard drive labelled “Preservation Copies 1 Working Machine”. This monthly folder should also be kept in the “archives” folder, as per the “Risks Inherent in the Scheduled Backup Software and the Creation of Preservation Copies” section of this document.
• Copy the “Current Backup” folder and any new “Preservation copy…” folders found on the external hard drive labelled “Preservation Copies 1 Working Machine” to the (second) external hard drive, labelled “Preservation Copies 2 Working Machine”, as per the “Managing Preservation Copies” section in this document.
• On the third (“Preservation Copies Archives”) external hard drive, copy the most recent weekly backup of the Archives database to a monthly folder on the same drive. Overwrite the any backup for the same month in the previous year, as per the as per the “Backing Up Content on the Archival Computer” section in this document.

To be performed on an annual basis:

• Starting immediately after the first backup on the working machine, Use MD5 Checker to copy the “Current Backup” folder and all its subfolders located on the external hard drive labelled “Preservation Copies 1 Working Machine” to a new folder named “Preservation copy backed up YYYYMMDD” (where YYYYMMDD is the date the backup was made) on the same hard drive. Ensure that MD5 Checker creates and stores MD5 checksum data for all files in this new directory, as per the “Risks Inherent in the Scheduled Backup Software and the Creation of Preservation Copies” section of this document.
• Starting immediately after the first backup on the Archives machine, use MD5 Checker to copy (along with checksum data) the most recent weekly backup (located on the third external hard drive) of the archives database to an annual “Preservation Copy” on the same external drive, as per the “Backing Up Content on the Archival Computer” section of this document.
• Test fixity of all preservation folders on all three external hard drives, as per the “Testing Fixity” section of this document.

To be performed once every five years:

Acquire three new external hard drives, test fixity of files in preservation copies of the current hard drives, copy all folders to the new hard drives, and dispose of old external hard drives, as per the “Refreshing Media” section of this document.