Reducing Costs of your BI Solution

i just read an article on searchcio (cio news) that talks about what to outsource, what not to outsource, and why.  it covers some major and interesting points about why you can/cannot manage outsourcing projects along the way.  for many years, i’ve heard about bi and edw projects costing too much, and taking too long, and being too complex — especially for outsourcing.  well, times are changing.  in this post, we’ll explore the cio rules, and how the data vault can actually help you take charge of your edw/bi project, so that most of it can be outsourced!

first, a link to the article in question

i think we can all agree that there is a need for more outsourcing of edw and bi projects, but we can also agree that it goes way beyond the project and extends in to the infrastructure, the hardware, the servers, and yes – even the data.  with the advent of cloud computing, and saas, and now paas (platform as a service), there is an even greater need to get your edw/bi projects moved or outsourced. 

keeping in mind of course that the best and brightest it assets you have must stay around to oversee the project, and guide it to delivery.  why?  because they are the ones that understand the business.  that is: the business keys, the business rules, and the business itself – they have an understanding about what the business needs that simply is near impossible to outsource (and shouldn’t be!).

so why isn’t the bi/edw outsourcing happening already?

 well, let’s take a look at the questions that the article raises, and address these issues one at a time.

1. can the it function be performed according to a set of rules and procedures?

this facet of an outsourcing strategy is the “bright line” between it functions that should be outsourced and those which are best kept in-house,…

the data vault methodology, along with the data vault model brings these rules, standards and procedures to the data warehousing/ bi world.  for too long, edw’s the world round have been built hap-hazardly, the industry hasn’t grown up, hasn’t provided consistency, doesn’t have a set of guidelines – all we have had is guiding principles (we all think we need a staging area, we know we need data marts) – but this is where the knowledge has stopped!

the data vault brings order out of chaos, and allows your edw/bi projects to answer this first question with a resounding yes!  those of you who are building data vaults can attest to this principle….  so, on we go.

2.   is the application pegged for retirement or being moved to a software-as-a-service provider?

well, if you really want your data warehouse to move to saas or paas models (because of cost savings), you can only do it if it was built properly in the first place.  yet again, standards, methods, and repeatable procedures are what’s needed so that an external provider can actually build what you want, and meet or exceed the specification.  this is the true path to data-warehouse-as-a-service, or biaas.

managed service providers make money by optimizing, or getting up to speed on, the application up front and reaping the benefit of that work over the length of the contract.

in other words, you cannot optimize what you cannot measure, and you cannot measure what you cannot monitor – so therefore, without standards, repeatable procedures, and processes to guide your project, optimization will be near impossible!

3.   is the application or it infrastructure unstable?

no hold on, who ever heard of an edw or bi solution being unstable in the infrastructure layer?  this never happens, does it?  (i’m joking of course).  you only have to look as far as the end of your nose, at the closest project (or most recent project) that was shut-down, re-architected, and re-started.  that is an unstable infrastructure!  perhaps the cause was: “need to adapt the infrastructure for real-time”, or maybe: “what we have architected doesn’t scale properly”,  maybe it was business driven: “well mr. business user, it will cost x and take y…” and the business responds by saying: “it costs too much, and takes too long”.

in other words, unstable infrastructure is driven because of the massive complexity embedded in the system!

 if it is something that can be fixed internally, you should do that first. the reason is that you don’t want to outsource an operation that is hugely sub-optimized.

the data vault comes to the rescue here yet again.  it has the principles, guidelines, and architecture to make the infrastructure stable throughout many years going forward.  it has a proven track record at 3 gigabytes, and 3 petabytes in size.  it has been used in hundreds of real-time systems, and helps you with productivity at every step of the way (just ask the dutch tax authority about scaling team resources).

in other words, get to know and understand the data vault and what it brings to your table.  find out that it can help you optimize your entire edw / bi project!

4.   is the application’s history of change control and change management documented?

here we go again!  no processes, no standards, no documentation, no change control, no management, and all of that results in no conformance.  it’s no wonder these projects get too complex to maintain within 6 weeks of being launched in to production.  lack of accountability on it’s side can cause all these issues to crop up.  well, get off your rear-end, and start succeeding. 

when an application’s processes are not documented, and you basically are turning over just the code to an outsourcer, the provider will have to reverse-engineer the application to figure out how to manage it,

who the heck wants to outsource an edw / bi solution just to have it “reverse engineered”?  furthermore, if you pay for reverse engineering, how much more cost and complexity will that introduce to your solution?

forego all of those issues.  the data vault gives it the ability to automate, optimize, and yes — even generate many of the processes underneath.  it keeps complexity down, and allows the team to focus on control, change management, and documentation.  in fact, 80% of the documentation can even be generated too!

5.   is the team assessing the application going to be affected if the decision is to outsource it?

now this is not something the data vault model nor the methodology can help you with, except to convince the it personell that they are necessary (as pointed out above), they should be focused on business rules, business understanding, and delivery.  that will only quell the fears as long as the project plan shows them as resources utilized during the process. 

on the other hand, the it team building your edw/bi solution does not like maintenance work, nor do they appreciate building mundain tasks over and over again.  so, if 80% of the mundane work can be outsourced, then the team will be happier, and of course much more focused on solving the business problems which is where they are most valuable to the business to begin with.

6.   does your organization have strong vendor management skills?

in this case, do your it resources (who will assist locally with the bi / edw project) have strong vendor management skills?  take this opportunity to train these resources, as they can be invaluable assistance to managing and controlling the project – along with ensuring that the outcome of the production system actually meets business needs.

 for companies with meager experience in managing an outsourcing vendor, moving to a managed services model is fundamentally different from managing the day-to-day lifecycle of an application — and more challenging

at the end of the day, it’s all about treating your employees (that you do have) with respect and honor, but at the same time – offering standards, rules, and procedures for everyone to follow.  these are the steps to great success in your next edw / bi project.

please let me know what your experience has been, i’d love to have feedback from you.

dan linstedt

Tags: , , , ,


  1. Tweets that mention Reducing Costs of your BI Solution -- - 2010/12/22

    […] This post was mentioned on Twitter by Deef. Deef said: RT @dlinstedt: Reducing Costs of your BI Solution: A short walk in to the business world about how and why… […]

Leave a Reply