Migrating Data into a Collections Management System

Data migration moves data from one location to another, one format to another, or one application to another. This often occurs when legacy systems are replaced by new applications that share the same datasets. Migration occurs when an archives introduces a new system or location for the data, such as a collections management system (CMS). 

Moving important data and decommissioning legacy systems can put stakeholders on edge, so having a solid plan is necessary. Before migrating data, understand the relevant information technology and technical skills required of the migration team, especially for a migration without vendor support. Ensure that the archivists and other stakeholders understand who is responsible for various migration phases and who is the point of contact. Having transparency in everyone’s roles makes the migration process run smoother. 

Before You Begin

You will want to make a copy of the original database to be migrated and use it as the source of the migration; doing so protects the system from any problems that might occur during the migration process.

Archivists will want to review their data for quality. Identify invalid records, duplicate records, and duplicate controlled values. Erroneous or irregular data will be either be carried over into the new CMS or, in some instances, block the migration process.

The team will need to establish a robust data quality rules-management process and brief the organization on the goals of the migration project, including shutting down legacy systems. They will also determine what data to move and assess the quality of that data before and after the move.

Importing is Iterative

A CMS allows for a variety of ways to enter, organize, and retrieve information. Many collections management systems have import applications—which enable the user to transfer data from an older CMS, a database program, another institution, Microsoft Excel, or Word files. Often the data must be converted into a form the CMS will accept, such as a Comma-Separated Value (CSV) or Extensible Markup Language (XML) file. Of course, every CMS has fields to input data, but the organization of those fields varies from system to system. The data conversion process involves mapping fields from the old CMS to the new one, rearranging the data if fields mismatch, and cleaning data.

Migration is iterative. Typically, a report is generated at the end of each migration, indicating errors or issues that occurred. Use the report to decide if you should remedy any problems found in the results in the source data or the migrated data in the new CMS. If you correct the problems in the source data, you can conduct the migration again.

Check Your Work

Select samples of records to be examined when the migration is completed. Make sure to represent in the sample both the most straightforward and most complicated or extensive records in the overall collection. Building and testing the data assures that the migration will be successful, while validating the data demonstrates that the migration has met requirements and that the data is viable for use. Once the migration has been completed, archivists should decommission old systems and monitor their new CMS for changes. 

Easier Work Moving Forward

Once the new system is running, continue to clean up the data and update your data standards and policies. Since the new CMS will be more user-friendly than legacy systems, processes for improving archival collection practices should be easier. Once you have progressed through the process of cleaning up and migrating data, maintaining good data—supported by standards, policies, and procedures—will continue to optimize access and use of your archival CMS.