IT question and answers

A Guide to Data Movement & Migration

As Information Technology evolves, newer requirements surface everyday. Todayís corporate houses handle lots and lots of data. For proper information dissemination and for economic storage purposes, the need for data movement and migration has been felt in the Information Technology arena. Todayís article is devoted to two concepts that has become cornerstone in todayís corporate I.T. environment.
Generally, data movement or migration refers to data transfer from a source to a destination. The terms data movement and data migration are used interchangeably in the I.T. world. There are several aspects of data migration and varies from one particular case to another. For example, you can make a complete copy of a file from one site (location) to another site and then delete the original. Again, in another instance, partial copy might be the requirement. The data transfer usually takes place between different storage media and even different computer systems (for example Windows and Linux). For practical purposes, the data transfer is performed programmatically in batches. This frees people from performing tedious steps over and over again. Mostly the need for data migration is felt when corporate houses change their computer systems and storage formats. Again, when two sub-systems merge (as in case of company mergers and acquisitions), data migration is needed.

Things to ask before you perform data migration operation

The I.T. manager should understand some basic questions before initiating the data migration operation:
  • Is the data movement going to be automated? If so, what is the procedure?
  • Is the data migration operation 100% transparent to the applications running in the system?
  • Is the migrated data 100% transparent and available when accessed?
  • Is the data migration procedure scalable? Is there any pitfall?
  • Whom to consult regarding technology and policy issues?
  • Is the existing technology capable to conduct cleanup operations including deleting moved files?
An I.T. manager should undergo some research on these aspects and get convinced on all relevant issues before performing the actual operation.

Some key terms relevant to data migration

  • Legacy data: Legacy data is the stored data existing in your storage media. This includes databases, spreadsheets, image files, scanned images of documents etc.
  • Data Migration: As already indicated, data migration is the process of importing legacy data into new media. The migration can be done either manually by moving files from one disk location to another, or making the whole process automated. The actual process depends on the nature of the existing system, and state and volume of the legacy data that needs to be transferred.  
  • Data cleansing: Data cleansing is the process of preparing the data before migration to the new system. Data cleansing becomes necessary when the architecture of the source and the target systems differ. Often the criteria for data storage in the old media differ significantly from the new system and the two storage procedures become incompatible.  Design flaws in the old system might as well become a necessity for data cleansing. The main purpose of data cleansing is to prepare the data for the new system.

Process of data migration

Some of the basic preparatory steps of data migration are as follows:
  • Analyze the structure and format of the legacy data
  • Analyze the structure and format of the target data (would-be data in the new system)
  • Perform field mapping between the source and the target data and data cleansing activities if necessary
  • Define data migration policy (say, automatic vs. manual)
The actual process of data import is a combination of manual and automation processes in most cases. In few cases, the migration process is either completely manual or completely automated.

A typical data migration process might involve the following steps:
  • Creating data extractions from the legacy system
  • Perform cleansing operations on the data extractions according to the mapping guidelines
  • Executing data import operation in the new system by activating the systemís data import feature
  • Verify sample data in the new system by comparing data reports to the old system
Some degree of automated data migration with varying transparency levels are offered by primary storage vendors. Network Attached storage (NAS) and Virtual Tape Library (VTL) vendors offer data migration facility. Some solutions migrate data within a single storage system, while others can migrate data across systems backed by heterogeneous technologies. Network appliance based and host server based solutions are also available.

The data migration process is highly intricate and should be given importance and the requisite time to accomplish the task. While acquiring a new system, the data migration activity should be kept in the mind and the right system should be purchased. Otherwise, a failed data migration activity can become a cause of grave financial concern and loss of business intelligence. IBM, EMC Corp., Innovation data processing, and Softek Storage Solutions Corp. are some of the leading vendors in the data migration market. 

Disadvantages of data migration

The following are the two disadvantages of data migration:
  • The data migration process is time-consuming and repetitive.
  • The process involves substantial cost.