Have you found yourself in a situation where a natural world or business key is not available? We all have. What do you do in this case? How do you construct a Hub? Is there a way in which to still build the Data Vault? All of these are valid questions, and over the next few months I’ll try to answer them. Let’s see though if we can’t offer a few suggestions in this posting.
As we teach in the Certification Class (http://DataVaultCertification.com) you will find that if you don’t have a business key to construct a proper Hub, that you need to rely on several other techniques. These techniques are listed in a progessive order below:
- Look for multiple fields that construct as much of a unique key as possible – then try to determine if these fields are actually descriptive in nature. If they are character based fields, they have a higher chance of being “fat-fingered” or entered wrong in the application than if they are numeric or date / time fields.
- Still can’t find a key? Look for a source system numeric identifier – yes, unfortunately a source system sequence number. BUT try to ensure that the application programmers have “exposed” this un-natural key to the business, if they have (either on a report or on an application screen) then their is a good chance that Business Users are actually using this field as a Business Key. If these source systems have NOT exposed these numbers to the business (they shouldn’t) , then there is a risk that these sequences may be “re-assigned” during full restore of the source database. This would wreak havoc on your Data Warehouse (Data Vault or NOT) unless you have CDC (Change Data Capture) on the source system.
- Still can’t find even a Numeric Key? Then you are out of luck… My question to you then & to the business is: HOW do you identify unique rows? What’s the business process for selecting a single row out of multiples for editing purposes? Identify the business process, and try to establish a mechanical method for choosing unique rows. By Mechanical I mean: something the machine recognizes, that doesn’t need human intervention/human decision making.
Ok – so this is a brief list of things you can try. The RISK of using numerical sequences from the source is potentially HUGE. Especially when cross-integrating source data into the same Hub.. However, this is somewhat mitigated (same key means different things) by separating Satellites by source system feed.
At the end of the day, there are more ideas around business keys, and surrogate keys that are taught in the class. Ideas I don’t have time to re-iterate here now, but maybe in the future.
This can cause huge confusion when you go to get data out of the Data Vault. All I can say here is: DOCUMENT THIS PROCESS WELL. Ensure you’ve documented the GAP between “what the business believes is a unique business key” and “what the source systems tell you is the real data”
Hope this helps, have thoughts? Wish to contribute a solution? Love to hear from you.