Yesterday I read (RED) a WHITE paper by IBM on Dynamic Data Warehousing (see "New White Paper" at http://www-306.ibm.com/software/data/db2bi/data-warehousing/) that was sent to us. It’s basically a marketecture document disguised as IBM’s view of the future of DW.
But it highlights that the industry is still struggling with the basic problems we’ve had in DW sinces the mid-90’s when Harvey and I first started looking at it.
The fact is we’ve come a long way but still haven’t cracked some of the most fundamental issues of DW (namely confidence in data consistency, data quality, the infamous: single version of the truth, silos, and now the need for realtime or near-realtime access to operational data for BI purposes) indicates that most organization either don’t fully comprehend the magnitude of the issues or are not READY to make hard decisions on how to solve them.
The hard decisions include, but are not limited to: The shift to shortening times between data capture & BI information delivery. The realization that multiple versions of the truth will be with us for some time. The (and I’ve waited this long to say it) … The sorry state of data definitions, data models & the associated business rules applied to generate BI results – aka metadata. And my personal mantra: Performance is always an issue.