Planning a data migration in 2018?

Last year, Experian commissioned Dylan Jones and the team at Data Migration Pro to carry out a programme of research looking at the current data migration landscape.

The output was a fascinating, ‘in-the-trenches’, account of modern migration practices that identified some clear trends that can lead to project success. This blog will give a good overview of the key findings, so read on for my highlights.

If you want to explore these trends yourself, you can download a copy of the research report here. You can also check out our research teaser video below.

A challenging climate of change

We work with a wide variety of organisations on an even wider range of data management projects. Frequently this requires us to support the movement of data from one system (or multiple systems) to another through our data management software and consultancy expertise – data migration.

During our interactions with clients we often see some of the considerable project challenges a Data Migration Lead will face, including: pressure to deliver the project on-time and on-budget, lack of senior stakeholder support and engagement, lack of coordination amongst project members and excessive risk in the proposed methodology, to name just a few.

If we then consider some of the wider macro-environmental changes occurring in the market, for example, increased regulation, growth of data volumes, and increasingly complex technology architectures – achieving success in a business transformational role can sometimes feel like a minor miracle.

Data Migration Research 2017

The modern data migration

The research highlighted several interesting and contradictory trends common in modern-day data migration activity.

  • Success criteria: At a project level, over two thirds (69%) of the respondents believe their project was a success. And yet, over half of all projects (54%) overran the projected timings, and nearly two thirds (64%) overspent the planned budget. Of course, ‘success’ is an entirely subjective view from the eyes of the project lead (and they are likely to want to describe their work as successful), whereas budget and timings are far more factual and revealing.
  • Scale: The report also highlighted the scale of modern-day migration projects. 60% of those surveyed were moving more than a million records. 61% of respondents were moving 3 or more systems, and 62% utilised a phased delivery model, rather than a big bang approach. Data migration is no longer a one-off legacy system move, but a complex, interlinked piece of ‘business as usual’ activity for many organisations.
  • Delivery: How migrations are delivered is also interesting. Our research found that only 30% were delivered exclusively in-house and that 64% of migration practitioners still rely on manual SQL coding techniques. Clearly, so much reliance on external support highlights the complexity of modern migrations. It was also interesting that such a large proportion of migration leads are still hand cranking the process, when there is an array of intuitive tooling in the market these days to do the heavy lifting. Our platform, Experian Pandora, being just one of them.

For a full breakdown of the research results, you can download your copy of the report.

Key principles of data migration success

Without giving the whole game away (as I’d like you to read the report) there were a number of clear factors that influenced the success of a data migration and the quality levels of the result. Here are two important ones:

1) Adequate project scoping

An evident factor was the amount of project scoping carried out pre-migration to remove assumptions.

One interviewee stated, “The initial scoping strategy of our multi-phased data migration was very poor, resulting in an inability to scope the information that the business wanted, which in turn created poor planning and forecasting.”

Clearly, the respondent was feeling frustrated by a rushed migration scoping exercise (a common problem on migrations) that will have undoubtedly lead to issues downstream.

2) An excellent data quality strategy

Another factor was respondents’ reliance on good data quality throughout the project. Where data quality strategy was rated as ‘excellent’ we found that 87% experienced a successful outcome.

Projects with a poor, very poor or non-existent data quality strategy witnessed a drop in outright success, only 55% of projects were classed as successful.

In addition, it was observed that a weak data quality strategy more than doubled the likelihood of a project only experiencing partial success.
I’ve only touched on some of the key research findings in this blog, but there is far more insight and commentary about building business buy-in, delivery approach and planning throughout the report.

Download your copy and then come and chat to us about your next migration project.