It’s time for me to jump back in to the theoretical aspects, and consider some of the deepest roots of the Data Vault model… That is: the natural world. I’ve long held the belief that the Data Vault is modeled (albeit a poor mans model) after the neural images of what we believe our brains look like. I have a hobby, as many of you may know, of reading and trying to understand the beauty and simplicity of the architecture. Yet the architecture holds depth and complexity – or is the function that holds these things? In case you’re wondering what I’m working on, this is a dive into the theoretical, the unknown (or my unknown as the case may be).
I’ve written it in the technical book, I’ve showed images. From what I understand, the way we think is a combination of the form (the diagrammatic model/form of the neurons, dendrites and synapses) and the function of the brain. Or as the case may be, multiple functions of different parts of the brain. Some parts of the brain are said to house memories, other parts, images of parents, other parts are said to deduce fight or flight. Of course there is the major separation of what we know: short term memory vs long term memory.
I believe, when we think, or are cognitively aware, we are constantly taking in input from our senses (touch, sight, sound, taste, smell, etc..), grabbing specific images, words, thoughts from the “memory banks” as it were, and then applying context to the memories. Using these individual building blocks to form a consistent and cohesive thought, one at a time. They say the brain is a relatively “slow” computer, but how then can it get to emotions, considerations, and feelings – or even complete thoughts so fast? No one really knows…. but one thing is for certain: the brain, in all of it’s complexity, combines form, function, and content – in parallel.
I think that when we build systems in the Data Warehousing world, we are building primitive (very primitive) content stores. Just like the brain, the content stores of a data warehouse hold data over time. The brain tries it’s best to remember, categorize and index (if you will) information by time. When you think of you’re 12th birthday, or your 8th birthday, these are both along the “birthday” index – or content/concept retrieval path. Now, it’s a matter of “time” – as in WHEN did the event happen?
The next question might be: “what was the weather warm or cold? or Did your cake taste good?
Of course, these questions are the questions that begin to lend context to the data or the information. But I digress….
I believe that if we can build a system, that recombines, form (data model), function (retrieval, indexing, parallelism), and context (learning, neural networks, patterns of association, probability scoring) in a self-contained component (like hardware) that we can actually make a machine that begins to “perceive” things about the world around it. I think the data model must resemble (in some way) the Data Vault, or to be more specific: a neural model, where the data is keye’d off of important events, and where it’s got hundreds of connections (if not millions) to other information around it. The Data Vault carries Links for these purposes. I believe that adding function to the mix is critical in order to make use of the data, know where it is, run the retrievals and updates in 100% parallelism, and of course finally, context. This must be a combination of historical data, plus the “learning pattern” that is taught based on a finite world, along with teaching the “learning system” what the model truly is and how to leverage it.
The scientists say when we learn something new, we form new neurons, dendrites, and synapses. When we connect or associate memories, the dendrites get thicker – the stronger the memory, the more vivid the memory, the thicker the dendrites. They say that Alzheimers patients suffer from memory loss because these connections (these dendrites) deteriorate, the patients can no longer connect the proper memories to form context around their ideas. They also say that neurons die off when not used, or when memory loss occurs.
All of these “features” of the brain make me believe that we can build a prototype of a perception system containing the Data Vault Model, and that the model (because of it’s nature) is best suited to dynamic alteration. In other words, when the system learns new things, receives new inputs, etc… it can create Hubs & Links & Satellites on the fly for storage. That the “stronger the indicators” and the “higher the confidence”, the more links can be associated with that information. In reality, I believe that the Data Vault Model lends itself to the beginnings of a self-optimization pattern, that the model itself can & should morph automatically, or optimize according to the world around it.
Now before you go jumping off the deep end, or quoting some obscure scientific reference to me, please be aware that this is just a thought experiment. So in keeping with this tone, if you have contributions or arguments against this, please voice them here by replying or commenting at the bottom of this post. Also note: that I am not a brain surgeon, nor a neurologist, nor am I a cognitive scientist. I’m just an interested and curious computer scientist who dabbles in the theoretical possibilities of arriving at a dynamic and self-sustaining system.
Just imagine for a minute what it might be like to have a truly back-office self-healing (not self-aware), but self-adapting historical data store or memory, capable of “spotting new associations” for us, presenting those to us as a mechanism for review, and through that review or human interaction, we teach and guide the system to do better the next time… What would that mean to you? Is this even interesting?
Curious to hear your theoretical thoughts….