This entry discusses the age-old-question of ETL performance – a Data Vault approach and a Traditional approach. If you have comments, thoughts, or other experiences I encourage you to add your COMMENT to the end of this posting. As you know, I’ve launched one-on-one coaching, this is the kind of knowledge you get within the walls of one-on-one coaching. But for TODAY – I’m giving you the answers FREE to show you the kind of value you get when you sign up for my coaching sessions. Contact me today: firstname.lastname@example.org for more information.
How does ETL performance compare to a traditional approach? Qualify
First off, there’s no “real” way to compare these things, why? Can you define for me what a “TRADITIONAL APPROACH” is? Every Data Warehouse I’ve ever worked on (except Data Vaults) all claims to have their own special sauce, their “we did it here because….” – no one has a traditional approach, and I wouldn’t even know where to go to get a definition for this. Anyhow, from the heart here are my thoughts:
The ETL / ELT performance for loading a Data Vault SCREAMS. There’s no better way to put it. On the right hardware (as close to a comparison as I can get) – a full Data Vault loading cycle might take 40 or 50 minutes total time, (50 staging tables, 1 terabyte incomming data) – where a so-called “traditional” (type 2 dimension + facts) loading cycle for the same data sets would take about 3 to 4 hours…. The Data Vault model is built on patterns. Put simply this makes it powerful. The Hubs, links, and Satellites isolate the work (divide and conquer) so that it can all happen at once. AND it can all happen in PARALLEL… I’ve blogged, and blogged, and blogged on this – and yes, inside my coaching area I’ve built answers to all the secrets, I explain why this works.