May 2016 | Data Quality | Data quality solutions | Managing customer data | Marketing Solutions

An obvious starting point is to understand the need for assessing data

Nowadays there is a far greater emphasis on creating the ‘data driven’ enterprise so this invariably places far greater focus on the quality of the underlying data.

I think it’s also fair to say that many business and IT leaders have been burned by delivering projects and services on top of poor quality data.

If we throw in the growing demands from regulators, there is a clear need in companies of all sizes and sectors to smarten up and start managing the quality of their information assets.

Ultimately, the goal of any assessment is to answer the following question:

“How closely does our data match our expectations, requirements and goals?”

Essentially, data quality assessment helps us understand the gaps between the current situation on the ground and where it needs to be.

For example, you may be implementing one of the following projects:

All of these initiatives typically involve multiple data sets, long information chains, legacy data and complex transformation logic – a breeding ground for defective data!

By performing a data quality assessment in these scenarios we can dramatically reduce the risk of project failure, deliver a better outcome for the business and of course keep the costs in check.

Who benefits from a data quality assessment?

Historically, data quality assessments would have been carried out by technical teams and the results interpreted by technical staff. These days it is quite different, both in terms of the skills used in creating the assessment and the profiles of those who consume the final information.

You will typically find that multiple stakeholders need to access and interpret your assessment results, such as:

  • Local data stewards may want to know which individuals are breaching local data policies and standard operating procedures
  • Legal and Compliance staff may want to know how data quality is impacting corporate compliance rules or external directives such as Basel III and Solvency II
  • Business leaders may want to understand how poor data quality impacts their strategic goals such as competitive advantage or customer service improvement

I touched on this at the start of the post. Most assessments are far too technically oriented. They try to apply a technology-centric approach by examining only the data and then inferring some kind of business relevance.

You need to have business people involved in the assessment and a clear business impact framework designed into your assessment process.

For example, many people assume that their data profiling results are the end-point in the data quality assessment process, the reality is, they’re just the starting point.

Adopting a business focused data quality assessment – the process

In the diagram above I outline a framework that has served me well for implementing a more business-focused data quality assessment. It may look a little different to the classic data profiling-based approaches so let’s talk through some of the points in more detail.

1: Understand your profitability drivers

For most businesses, profitability is the primary driver for a data quality assessment which is why I focus on this at the start of the assessment. If your challenge is regulatory compliance or some other driver, by all means substitute what works for you.

My point here is that there is little point focusing on data that does nothing to drive the bottom line. For example, I once spent a week assessing data on equipment that was due to be terminated in eight months time! Are you assessing data that really has an impact on the financial performance of your organisation?

2: Build a Financial Performance Model

Most data quality assessments start with the data. They examine attributes, tables and relationships to build up rules for the assessment.

We still need to do this but I contend that the most important activity is understanding how our business functions first. This activity builds upon the first step, understanding where our profits come from (or any other driver the business are interested in).

In this stage we need to be finding data that tells us exactly how we’re performing, both financially and from a performance perspective. For example, how many site visits do our engineers make? How many customer calls do we process? What are the costs involved?

By gathering this performance information we create much needed context around the impact of data quality defects.

3: Build an holistic data quality system

This step is the crux to the whole process and the main reason I was able to create so much more impact in my data quality work. By combining data quality metrics and business focused metrics we can share data impacts in terms the business can relate to.

For example, in one utilities firm I was able to illustrate how defective data was directly increasing the cost of engineer visits simply by merging these data sets together into a unified repository.

In another project we showed the impact of having incomplete contact data and how that translated to lost opportunities in real financial terms. By linking sales, marketing and contact systems, with a data quality assessment, we could visualise the impact over time.

When you merge performance data and quality metrics, a powerful story unfolds.

You can use the Experian Pandora Free Data Profiler to get better insight your data and to pro-actively discover trends in quality and performance.

4: Create an interrogative reporting layer

The final piece in the puzzle of business-focused data quality assessment is to create an environment where the business can ask questions.

For example, in one firm, that had regional locations, I was able to overlay each location on a UK map and report on the data quality levels at each location. I presented the findings to an executive steering committee that became instantly engaged in the findings. Lots of heated debates and questions followed but by having the data at my fingertips meant that I could drill into the low-level results from those high-level business questions.

We have moved on from Powerpoint and Excel spreadsheets. If you want the business to get engaged in data quality, you have to create an engaging experience and the way you visualise and report your data quality findings is pivotal to this process.

In my next post I’m going to walk through an example scenario of how all these pieces fit together to create a business focused data quality assessment.