Real-Time Business Intelligence: Technology and Strategy Implications

In an earlier article we discussed how increasing competition has led to the compression of the business intelligence life-cycle from a monthly to an almost continuous frequency in some cases. This challenges technology vendors and business strategists alike to respond to the new reality.

Technology Implications

When businesses updated their BI data monthly, processing of the data could be conveniently scheduled outside business hours. A processing window would be established (say, from 6 PM to 8 AM local time) for the data architects to extract, transform, and load (ETL) data from the transaction processing system such as sales orders or general ledger to the BI system. As demands for amounts and detail of BI data increased, it became more difficult to keep the processing within the allotted time window. But as long as this meant management occasionally had to wait until 10 AM or noon local time for monthly updates it wasn't a major problem. The flaw in this strategy was exposed by demands for more frequent updates.

Clearly what is needed from a technology standpoint is a more targeted and incremental approach to processing data. For example, a Microsoft SQL Server Analysis Services database (cube) can be programmed to monitor for changes in its source transaction system. After a change is detected, the cube can "listen" for a pre-determined amount of time to make sure there are no more changes to the source data, and then issue a request for only the new data. Since this represents a very small amount of data from the source system, and the portion of the cube that needs to be reprocessed is also quite small, the BI data update takes seconds instead of hours as in the old processing strategy. Other vendors are building similar capabilities. Another technique involves the use of database caching to constantly process changes to data in the background while continuing to present the latest available view of the data to the consumer.

Strategy Implications

Elsewhere on this site, I have advocated for a BI strategy that is in harmony with the way people interact with data. I introduced I concept I call the PRIME model, with five activities: Producing, Reviewing, Investigating, Monitoring, and Extrapolating. For more specifics, please feel free to click here. Based on the PRIME model, I can foresee three significant impacts on how businesses work with data as opposed to the current approach.

First, real-time BI and analytics will shift the focus of data consumption for knowledge workers, analysts and managers alike, from reviewing to monitoring. With a greatly reduced time window for the consumption of data before fresh data arrive, there simply won't be time to review and digest static reports. This will lead to a greater emphasis on business performance management. The traditional report will increasingly be boiled down to the presentation of a small number of key performance indicators (KPIs), along with strategic drivers of those KPIs.

Second, because data updates will occur too rapidly for a human analyst to investigate causes or extrapolate trends using traditional methods, there will be an increasing dependence on automated technologies for data mining and predictive analytics. The investigation and extrapolation performed through such automated technologies will feed into the monitoring I described in the last paragraph. These automated technologies will not supplant but will augment the knowledge worker, whose focus will shift toward longer-term strategic analyses.

Finally, the "24-hour BI cycle," as I described it in my previous article will drive an increasing reliance on mobile business intelligence, as managers and analysts require access to data wherever they are when business conditions change. IT will need the ability to push needed data to these knowledge workers, whether the client machine is a traditional PC, laptop, netbook, smartphone or tablet. It's not difficult to imagine that, over time the latter two devices will play a greater role as their capabilities and the wireless network infrastructure continue to improve.