Hello everyone, I am bringing you the exciting news: If you want your Data Vault 2.0 to be 100% insert compliant (as it was in the beginning, way back in 2001) then REMOVE last seen dates, and REMOVE load end dates.
I’m here to tell you, that if you aren’t focused on an insert only architecture for your Data Vault 2.0, then you aren’t focused in the right place. Due to scalability (volume), and sheer arrival speed (velocity) of data, your Data Vault 2.0 model should be highly focused on 100% insert only architecture.
Last seen dates were always a kludge, a crutch, to help developers get around “full table dumps” and detecting data that disappeared. Load end dates were always a kludge, a crutch, to help developers issue between clauses. on query access.
Well, I’m happy to say: there is a way forward without either of these two kludgy attributes.
- Implement Record Source Tracking as defined in my book: Building a scalable data warehouse with Data Vault 2.0, OR as defined in my CDVP2 class. What??? You’re not yet certified? Why? What’s stopping you? I am offering two additional courses this year, you can sign up for them at: http://DataVaultCertification.com
- Implement Point In Time and Bridge Tables – AND DO IT PROPERLY!!! Again, as defined in my book: Building a Scalable Data Warehouse with Data Vault 2.0, OR in my CDVP2 class.
By the way, the benefits of properly implementing these two paradigms far outweigh the costs of “updates” against the Data Vault model. Furthermore, these two structures move your entire EDW DV2 in to the 21st century, and make it 100% insert only compliant.
Moving to 100% insert only architecture is currently (and in to the future) the best thing you can do for your EDW DV2.
If you have questions or comments, feel free to post them below.
(C) Dan Linstedt, 2016 all rights reserved