What are the active measures I need to put into place to address issues around data archiving and storage?

This guide is designed to help leaders of heritage organisations decide on the best storage option for archiving digital data. It looks at what you need to consider before you choose a storage solution, as well as the pros and cons of some of the main options available.

Various items such as mugs and lights on display in a museum
Image courtesy of Heritage Cities ©

What are the active measures I need to put into place to address issues around data archiving and storage?

1. What is data archiving and why is it important?

Working in the heritage sector, you know that some of the data your organisation creates today will become tomorrow’s historical artefact. But creating data is the easy part – you need to be able to find, access and use that data in the future.

The problem is that digital data is fragile, as are the systems used to store it. You’re probably familiar with how to preserve physical files, but without a proactive and long-term strategy for storing and archiving your digital files, they won’t last for as long as you need them.

In this guide, our information specialist, Elissa Truby, will show you how to choose a suitable storage solution for your data based on:

  • how long you want to keep it
  • how you want to access it
  • whether you have the resources to manage it yourself.

She will also explain how to select data for archiving and how to document your decisions.

What exactly is data archiving?

The technology industry has borrowed the term ‘archiving’ from the heritage sector and changed its meaning slightly. For example, when you ‘archive’ an email it means moving it out of your inbox, not that you’ve done enough to preserve it forever. Archiving is different to backup, which means making copies of your data for disaster recovery, although backup also plays an important role in protecting your data.

Data archiving here means moving the older data you want to keep for future reference from the area you use to store your current work to a location designed for long-term preservation. Separating the storage of your current and archival data will help you to:

  • reduce storage costs by keeping data you don’t access often on less expensive media
  • free up space in your active work area to increase efficiency and productivity
  • protect your most valuable data from accidental modification or deletion so that it remains authentic over time.

2. Steps towards storing and archiving your data

Step 1: Understand your data archiving needs

The first step is to identify the different types of data your organisation creates and collects. Not all your data is equal – think of the number of emails you send a day for short-term activities. Could it be that some of the data you’re storing is no longer useful and is only taking up space?

At the same time, all companies need to keep certain types of data to comply with the law. You might only need to keep this data for a set time without needing instant access to all of it.

As a heritage professional, you also want to consider the lasting historical, cultural or social value of your data, and preserve some of your data for future generations or share it with external researchers. Many heritage funding bodies will ask for evidence of how you plan to store your data over time before they’ll award financial support.

No matter what data you have, think of your archive as a treasure trove of information, not just ‘old stuff’. Focus your time, effort and money archiving your most valuable data.

Step 2: Choose your storage solutions

No storage option is a permanent fix because no technology is built to last forever. That’s why digital preservation experts recommend using a range of storage mediums in case something goes wrong. Whichever option you choose, you need to regularly monitor your storage hardware and have the resources in place to manage and maintain it – both in terms of budget and technical skills.

There are several storage solutions available depending on your organisation’s needs and several are listed below:

Cloud storage

Cloud-based storage is a popular choice for data archiving because you can access your data quickly over the internet wherever and whenever you need it. This works well if your organisation functions across multiple sites, if staff and volunteers work remotely or if you want to provide external access.

Your data is stored offsite and managed by a third-party. Many providers offer built-in archiving services. In fact, several commercial digital preservation and repository systems use cloud technology. This makes cloud storage a practical solution if you don’t have technical staff at hand.

Cloud storage is usually cost effective. You can store large volumes of data and grow your storage capacity as needed. You’re normally billed monthly for the amount of storage and services you use but beware of extra charges for data retrieval.

A big drawback of the cloud for archiving is that by outsourcing responsibility to a third-party, you lose some control over your data. Always have an exit strategy so you can get your data back should you change your mind. Select a trusted provider who has experience of working with heritage organisations like yours and who understands digital preservation principles.

Local disk drives

Disk storage is a type of onsite storage that’s attached to your organisation’s network. There are different types of disk drives available, which vary in speed and cost. If you already use network drives, you know it’s easy to directly access your data, organise it and find it.

However, disk storage can be expensive, especially if you have a lot of data or very large files. Disk drives need to be kept in a climate-controlled environment, meaning high energy costs and regular cleaning. They also take up a lot of physical space, which isn’t always a luxury available to heritage organisations.

The biggest downside of disk storage for archiving is that the hardware has a short lifespan and can fail suddenly without warning. Disk drives need to be managed by staff with IT knowledge who can monitor the hardware for preservation issues and migrate data as systems age.

Tape drives

Tape drives are storage devices that use magnetic tape to read, write and record data. Your data is normally stored offline, but you can attach tape drives to a computer. Companies have used tapes for archiving since the 1950s, so it’s considered relatively reliable. It’s hard to modify data on tapes, making this a highly secure option.

The main issue with tape storage is that it can take a long time to access your data and it comes with limited search and indexing capabilities. This means tapes aren’t suitable if you need instant access, and you should be extra careful in documenting which tapes you use to store what data.

Don’t be fooled by tape’s reputation as a guaranteed solution for long-term preservation – no such thing exists! While tape drives normally last longer than disk drives, there are still risks of damage and data corruption.

Step 3: Create a data storage and archiving policy

Once you’ve looked at the different types of data you have, and explored your options for storing your archival data, develop a policy which sets out:

  • criteria for what to archive and where
  • the type of storage media you’ll use
  • responsibilities for archiving data and managing storage
  • rules for data access.

3. Set your strategy in motion

Now you’ve learnt what data archiving involves, and considered some of the options for storing your data, reflect on this by downloading our ‘Questions to help you choose the right archival storage solution’ (PDF file, 329kb).

Further resources

You can find out more about digital archiving and preservation by visiting the Digital Curation Centre or Digital Preservation Coalition and reading their useful guidance.



More help here


Book on a scanner

I’m a small-sized heritage organisation, how do I plan for digitising my content?

Digitisation serves a range of purposes and offers many benefits for heritage archives. As a small organisation you may be wondering how you might plan and prepare for digitisation. This is a concise guide on how to get started and what to consider.

 
Grey filing cabinet with an open index card drawer.

Where do I start with creating a digital archive?

Creating a new digital archive requires a bit of forethought and consideration. In this article, we’ll give you some useful tips to help you get started with the process of creating a digital archive.

 

Browse related resources by smart tags:



Data Data collection Future Future proof guide
Published: 2022
Resource type: Articles


Creative Commons Licence Except where noted and excluding company and organisation logos this work is shared under a Creative Commons Attribution 4.0 (CC BY 4.0) Licence

Please attribute as: "What are the active measures I need to put into place to address issues around data archiving and storage? (2022) by Elissa Truby supported by The Heritage Fund, licensed under CC BY 4.0




 
 


More help here



Digital Heritage Hub is managed by Arts Marketing Association (AMA) in partnership with The Heritage Digital Consortium and The University of Leeds. It has received Department for Digital, Culture, Media and Sport (DCMS) and National Lottery funding, distributed by The Heritage Fund as part of their Digital Skills for Heritage initiative. Digital Heritage Hub is free and answers small to medium sized heritage organisations most pressing and frequently asked digital questions.

Arts Marketing Association
Heritage Digital
University of Leeds logo
The Heritage Fund logo