DW and Big Data

One of my specialties is large warehouses and the new trend of big data maps very nicely into this.

My basic architectural argument is that you should target data warehouse products to systems that HAVE a data model and that Big Data is best used where the model is more loosely defined and near schema-less – which include many of the data structures traditionally used for auditing and history tracking.

If you think of the data flow at a very high level, from a source to the warehouse, this is my approach:

image

I have written quite a few blog posts that I hope you will find are coming together in a coherent theme of design guidelines. To make it easier to locate them, I have provided the index below.

Keys and how to handle them

In order to deliver clean and integrated data, the proper and high performance handlings of keys is crucial. I have blogged extensively about this here:

Warehouse Knowledge

Information about large databases and how they work.

Myth Busting

I like to put “common sense” recommendations from the database community to the test. Here are the posts I have written about this so far:

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s