BI Concept: Master Data Management (MDM)

When I first introduced readers of this blog to business intelligence I claimed that, by definition, BI had to be "integrated and coordinated" in order to be successful. I gave the example of the boss who, confronted with two or more sets of sales data, wondered in frustration which was correct. I could just as easily have made the example about the boss wondering which of several customer or product lists was correct. I made further allusions to this concept in discussing capture of data as the first step in the BI data life-cycle. And this is a major consideration in using extraction, transformation, and loading (ETL) tools to clean up such problems.

Unfortunately, this cleanup is not always fed back to the front-line systems where the data are initially captured. And there may be many such systems: lead generation systems, sales order systems, accounts receivable, and so on. And in most cases front-line workers are looking at the transaction systems they use daily, not the data warehouse.

This problem is the impetus for master data management (MDM). Simply put, the goal of MDM is to ensure that the view of company data is uniform across all business units and systems. There are two pieces to this. The first piece is to create a master data set for each of the critical dimensions (such as Customer or Product) of interest. The idea is that by referring to the master data set all parts of the business have a unified view of business data. This is depicted in the slide below.


This sounds like a terrific idea, but it leaves your front-line people looking in two different places, their data entry system, and the master data, for information they need. So the second, and much more difficult, piece is to integrate the master data back into the source systems, for example as in the next slide.


There are several approaches to doing this: by programming the source system to always look at the master data, or by actually re-writing source system data with master data, or any number of other strategies.

Sounds like a lot of work!

The notion of trying to tie every major element of interest together across all business units and systems can sound daunting. Many businesses choose to focus on specific areas of the business that they feel are most critical to their success. Thus you'll hear about Customer Data Integration (CDI), or Product Data Integration (PDI), or variations of such acronyms. Think of these as subsets of MDM that are focused on those specific subject areas.

Who are the players?

As you might imagine, many of the major vendors in BI in general are well represented in the MDM field. These include IBM, Oracle, Informatica, and SAP. But there are some lesser known companies in Gartner's "Magic Quadrant," among them Tibco, DataFlux, and VisionWare.

The Data Integration Imperative

Consultants and experts in the BI field use the terms “data governance” and “data integration” to talk about how to approach the kinds of problems we’ve been discussing in this section of the material. These fundamental concepts lie at the heart of the “integrated and coordinated” part of the definition we gave earlier for BI as a whole. Previously, we introduced data governance in two articles. This article introduces the concept of data integration.

Whereas data governance (in my opinion, at least) is really a people concept that requires a human touch to manage properly, data integration is more a technical concept for implementing the parts of data governance policy that call for BI to reflect the business as a whole. Put another way, data integration is the activity of pulling together data from systems all over the business and tying it all together so it says something meaningful about the whole business.

If you’re just starting your own business, or you’re in a business unit of a company where there is little or no data governance in place, data integration is simply not a high priority. You may only have one set of data to work with, as was the case in “'Real' World Story #1.” In that case the output was meant strictly for internal consumption by the sales force. In the case of the fictional specialty retail store we discussed earlier, the only data available in the beginning might be point of sale data plus some cost data.

But as the business grows in size and data accumulate in more and different places, data integration becomes more and more important. As our specialty retail store develops, sales are collected both in the store and online. Also, there may now be shipping data sitting in a completely different place. Without some way of tying all of this together it becomes difficult to impossible to get the big picture of how the business is doing. So whether you need it or not to begin with, it’s never too early to start thinking about and planning for data integration in your business.