Data Vault: About the Standards And Methodology

I wish to address a matter of importance to me, and while this is not a particularly pleasant topic, I hope that you will respond – by replying to this blog entry and letting me know your true feelings on the subject. This entry is about the Standards, Certification, and the Methodology.  WARNING: part of this is a rant, and it may in fact offend you.  Please read with a grain of salt, and put yourself in my shoes – try to think of this from my perspective, then, please, respond to the post by adding your comments.

First, about the standards…

I could be wrong, but it seems as though there are some people out there in the community who believe they should be the ones “changing, stating, and documenting” the standards for the Data Vault Modeling and Methodology.

I’m not opposed to suggestions to changing the standards, but I must say, that it is absolutely vital that if any changes are to be made, that I have the final call on this.  Otherwise, whomever is proposing the changes may actually re-invent the wheel.  In other words, there’s a reason why it took me 10 years of research and design to get the patterns right, I’d hate to see these folks go through the same pain I’ve got – only to break the model, break the standards, all for the sake of change.

Now, that’s not to say you can’t propose changes to me, I think you can, and you should – this is how standards evolve over time.  I am saying that changes you propose should follow the rules of a scientific experiment.  They need to be tested – for performance, durability, stability, pattern analysis, maintenance and complexity ratings, etc…  There’s a whole set of tests and procedures that should be undertaken before changes are approved.

I will continue to maintain the tried and true standards of the Data Vault methodology, however I am open to suggestions – but only after they’ve been exposed to a number of tests, and proven themselves to be enduring.

Regarding standards, I’m in the process of creating a “standards board” – involved with individuals who are a) trained, b) certified, c) have proven themselves in the field with experience, and d) come to the table with a wealth of background.  While I will have the final say, as well as contributing power on this board, this board will have the power to document and release new standards, and recommended procedures moving forward.

By the way, I maintain a copy of the over-simplified standards. The rest of the standards for the data modeling components have been written in long-format in my technical modeling book, at: http://datavaultcertification.com

And here’s where Certification comes in:

I will not recognize ANY other party who claims to be teaching Data Vault Certification classes, unless they are listed on the authorized trainers page at http://DataVaultCertification.com

This means: no other classes, organizations, affiliations, knowledge bodies, or groups will officially be recognized as certification authorities for Data Vault tests and certifications without approval by me.  If you know of an organization or set of individuals performing or claiming that they offer certification, and they are not sanctioned or authorized, then they are not fit, not suited to provide you with anything other than a “certificate of completion”.

More Standards and Now, The Methodology side…

You may think for a minute that the Data Vault is nothing more than data modeling, and you would be wrong.  It’s much richer than just data modeling… It includes a philosophy of implementation, a project based methodology with it’s own rules, it’s own standards, and it’s own mechanisms for “getting your EDW right, THE FIRST TIME!”

I’m sorry to stand on my soap-box here, but I’ve been around the block too many times, seen too many EDW’s “fail” or not make it, or not cut the mustard, or not be what the business wants – all to be shut-down and re-started, re-architected, re-built, outsourced… you name it.

When I built the Data Vault Model, I built a project methodology around it, based on SEI CMMI Level 5 principles – that’s why there are SOOOO many patterns embedded, that’s why it’s repeatable, that’s why the metrics of complexity are very very low, that’s why the implementation phases are generally quite quick, painless, and easy to do.  It’s also why there is software available for “generating” loading code patterns.

I can’t begin to stress to you the actual importance of the methodology, AND FOR THOSE OF YOU CONTEMPLATING CHANGES TO THE STANDARD STRUCTURES IN THE DATA VAULT MODEL – DON’T!  Why?  Because unless you consider the methodology side of the house with your changes and the impacts to the loading, complexity, management, and maintenance, then you will be doing a huge dis-service to the Data Vault Community.

You simply cannot “make a change to the foundational or core architecture” of the Data Vault structures WITHOUT considering the impacts on the methodology.

Now where’s one more thing that might twist your knickers in a knot…

I’ve not yet written very much about the standards of implementation…  There are very few individuals out there who “get it”, or “got-it” and understand what I’ve done.  Those that do have it, know just how powerful the methodology is, and how powerful it can be when the standards are adhered to. I have great respect for these individuals, and it is these folks whom I will invite to become a part of the Data Vault standards governance board.

The fact is, unfortunately, due to time limitations, I’ve not written enough about the methodology side of the house to provide for judgements about standards changes, and for understanding the impacts to the implementation methodology.  All of this is coming, trust me, I’m working hard on releasing more information that will help expand your knowledge – but I CAN’T DO THAT if you’re trying to change the data modeling structures and standards underneath…

Anyhow, this isn’t a knock on anyone, it’s simply a wake-up call.   Changing standards, or even suggesting such things is not something that should be taken lightly, and of course as always…

BUYER BE-WARE.  If you run into individuals, or writings, or authors of such statements, and they don’t have my seal of approval – then you probably should dismiss their claims as being bogas, or not completely thought through…

Am I being a stick-in-the-mud?

YES, and this is because every system I have ever built on the Data Vault (model & methodology) and every system I ever will build, I will stand on, stand with, point at, and say: I believe in it, it works, I guarantee it.

When was the last time you heard a guarantee from your IT staff?  let alone the consultant who is proposing “standards changes” to a modeling and methodology system they didn’t create?

I am sorry if I have offended you, but you (and you know whom you are) have continued to try and hide this fact from me.  I am willing to work WITH you to entertain standards changes, but come to me with proven test cases, not just “one-offs” or single instances where “this worked, and this didn’t”  I need proof – scalability, real-time, maintenance, complexity ratings, the whole 9 yards…

So you STILL want to change the standards?  Great, let’s work TOGETHER – because teamwork is better than individualism, and more heads are better than one, and Yes, I believe there are still great things yet to be done, but PLEASE don’t throw the baby out with the bathwater by trying to go around me, forming your own “group” to create standards that may not cut the mustard.

Tags: , , ,

3 Responses to “Data Vault: About the Standards And Methodology”

  1. Kent Graziano 2011/02/15 at 9:01 am #

    Dan,

    I am behind you 100% – after all you INVENTED Data Vault! Anyone proposing changes to the architecture without working with you is out of their minds (IMHO).

    Folks – if you don’t like all of the Data Vault patterns, methods, or standards, that is fine. You are all intelligent, educated, and experienced and have the right to disagree and seek alternative methods. If you are not going to follow all the standards then you are not implementing a true Data Vault. That is really okay. But please be honest with yourself and your clients about what you are doing. Call it Data Vault-light (or something). Tell them your are basing your work on Data Vault (give credit where credit is due) but that you have decided not to follow all the standards (and why), then after you have long gone from the client they have a fighting chance of moving forward with the least issues. Then if another DV certified consultant (or even Dan) gets called to help them out, everyone will know from the start that they are dealing with an un-certified varient of Data Vault (and alter their rate/bid accordingly).

    But really – why would you just not follwo the standards? 10 years of R&D is a pretty compelling reason, I would think….

  2. Stephan Deblois 2011/02/15 at 4:53 pm #

    Dan,

    I agree with you 100% – the top authority on the Data Vault methodology is you. I don’t know about others but for me, the increasing level of noise around the Data Vault is remind me of that other data warehousing civil war we all went through 10 or 15 years ago (Inmon vs Kimball). I almost never bother to reply/comment anymore, I don’t feel we are going anywhere. People break the standards and then argue for weeks about it.

    Like Elvis said: “A little less conversation, a little more action please”. 🙂

    There are still some (very limited) areas of the methodology that I am not comfortable with but what we have is extremely powerful (and many times better than anything else out there) and I know that you are the person to go to for evolving and perfecting the concept.

    If I have to “play” with a standard (which I very rarely do) I do exactly what Kent is talking about. I let my clients know that what they get is inspired by your work and I publish to everybody the official web sites and information about the Data Vault so they can explore and understand the tradeoffs we choose to make.

    All I have to say is that I am, today, making clients very happy by designing and developing data warehousing solutions that are flexible, scalable, generated and automated. This would not be feasible without the Data Vault methodology and all the R&D time you have put into it.

  3. Kevyn Schneider 2011/02/16 at 10:28 am #

    Dan – I too agree with you, Kent and Stephan. In the past I believe everyone saw you personally as the standards bearer as you are the DV inventor. If others are now thinking of taking on that mantle, they should give it a new name as it is no longer true DV. That said, you really do need to create a standards body of some sort – before they do. While I’m sure you’ve already done this, I would advise that you review their ideas carefully and include what can be included in your standard and document why a particular idea is NOT part of the standard if that is the case.

    Now, as was discussed at a DAMA meeting last night, standards are a great thing which we all love to bastardize and extend to meet the needs of specific organizational realities. Many many moons ago we implemented a modification of DV at Administaff. Since then I’ve been a huge DV fan, but would never refer to our design as “standard DV”.

    I see no problem with others doing what we have done – bastardizing a standard to meet their own needs. But they need have the humility to understand that their bastardizations do not redefine the standard.

Leave a Reply

*