In the world of Data a good warehouse is like "Plumbing" someone once said to me. They then took offense to their own words saying that it cheapens the value.
I find myself in many executive conversations of late where people are having to defend the purpose and value of their large data stores (specifically Enterprise Data Warehouses). I have often heard many different reasons, but it largely has to do with being able to find and get to the right data to run your business – releasing the value that is there from the data that is there (no not finding hidden meaning).
I was in a meeting a while back with a top 5 automotive manufacturer who said their BI tools didn’t work to respond to the business, because they could not find the data to report on. Another Fortune 100 client of ours often states that they can ask their team a question and get 5 different answers back. So how do you quantify the answer into value saved by going through the expense of a large data warehouse.
Jeff Jonas had a great post "When is a Data Warehouse not a Warehouse?" recently that twigged my attention and I want to include a piece here as a backdrop to some further conversation, because his examples of supply chain in retail "Warehouse vs Mart" are great from a technical point to kick off.
"This is a warehouse (this picture is worth a 1,000 words). More specifically, a Wal-Mart warehouse located in the middle of nowhere on Interstate 15 between Las Vegas and Salt Lake City.
Warehouses are about strategic distribution. They
are engineered to support three primary functions: (1) a receiving
function; (2) a staging function; and, (3) a distribution function. Ideally,
warehouses are strategically located, i.e., physically constructed in
areas where expansion is economical, convenient and located in
proximity to efficient distribution channels (think highways or
railways). Warehouses are designed to support everchanging inventory
requirements (e.g., from pet rocks to tandem bicycles). Their inventory
is organized towards maximizing efficiency at scale (e.g., pallets and
forklifts). And they are appropriately secured
(e.g., protected by a fenced perimeter and a guardhouse which controls
the arrival and departure of product).
Marts (picture this), on the other hand, are located and engineered to serve users. They are conveniently located and readily accessible (e.g., on site parking). Content is highly predictable – consumers know which marts have what product. Inventory
is organized in a manner best suited to the products offered and
customer expectations. This is why Starbucks, Kroger and Payless shoe
stores all have unique and highly specific inventory models (picture this). Product
is often presented in a manner designed specifically to drive
consumption, and frequently optimized towards guiding consumers towards
product with higher margins. Marts are secured according to the value of the content – that is why pharmacies are secured differently than 7-11’s."
I am not going to add any more of his stuff, though the above is a lot. I really suggest you check out his post. But I liked this and would add the following:
Business Intelligence, I use this to reference the applications that take the data and turn it into something meaningful for the end-user. This is where the real value comes to the business not the the other layer. So back to my question – Why Warehouse Your Data?
The business does not have the time or usually the knowledge about the data in their source systems to be able to derive the value from their Business Intelligence and Analytics tools.
So looking at the example above, Data Marts as necessary (though Big Box stores are doing a good job of killing marts) is one way to get specialized and very focused data sources out to the user. But if they want TV’s at 7-11 they are out and their data is not sound. Even the Marts that are smart and have scale have distribution hubs or warehouses that hold the products for fast distribution to the marts. So marts don’t generally in high productive world pull from the sources.
Now we are talking about Warehouses or also Big Box stores as an analogy for letting the customer have everything right there for them to pull what they need. So if I take a liberty here and refer to a big-box plaza as a good example of an Enterprise Data Warehouse (many segments) then I can go to Wal-Mart or Warehouse and get what I need across many segments.
So when size of data and organization fits, then having a centralized, receiving, staging and distribution makes sense – and thus a Datawarehouse. And when performance is OK, then make the warehouse the store and let the people get what they want, if that isn’t working then add local stores with more specialized selection for the market.