I would like to share some thoughts on the top opportunity and challenges that Business Intelligence and Data Warehousing will face in 2011.
1. Mobile BI – As with any technology or approach at its genesis, organizations often struggle to find its relevance. What should an organization evaluating Mobile BI be looking at to determine if, when and how to deploy? The opportunity to make information more available in our very mobile world is very captivating but it needs to be tied to the overall strategic plan.
2. Skills Shortage - As demand for IT services continues to heat up there will be a skills shortage in some areas as the year progresses. BI seems to be in strong demand so it follows that this could be an issue. Most BI technology is so new it is difficult to find experts with more than a few years of real in the trenches experience. This will mean longer lead times to start critical projects as well as escalating costs.
3. Shrink the Batch Window - With DW and BI maturing, rebuilding and upgrading are becoming significant activities. It's no longer all new. Now there's maintenance, support, care and feeding. Most of the maintenance is because the batch window is too big and needs to be optimized and shrunk.
The approaches we used in the past to move data have to change to accommodate the more frequent movement. Reports that once answered the questions we were asking must be changed or rewritten to evolve with the business. All these factors drive the need to rebuild our aging EDW / BI environments. These requirements must be factored into planning for 2011 and beyond.
4. Data Quality - Traditional Data quality issues from sources drives the need to change business processes related to source system data capture, business practice improvement, and system of record data accuracy. In addition, integrating external data (feeds from partners or for-profit data providers) creates another challenge as each organization applies different standards to how they manage their data.
5. Evolution to Real Time - The lines are blurring between real time systems and real time DW/BI. As BI becomes more real time the lines between BI and OLTP blur…when do they become one again? When do I go to BI instead of OLTP systems or vice versa? There are technical, business and political reasons driving these decisions and the evolution. This evolution will radically change the architecture and design of DW/BI systems.
5. Social Data – With the amount of good data that is available real-time, people are starting to look how they can use this data. There is a belief that the data relevance is high and so can be very actionable.