Avoiding BI project pitfalls

We've looked at BI roadmaps, and discussed ways to progress and build valuable decision support systems.
Lets take a pessimistic approach and look at what can bring a business intelligence solution to its knees before it has even got off the ground, despite a perfect roadmap. In other words, let us take a look at what to avoid, and how to best handle the issues that surround BI solutions, with some advice on preventing these conditions.
For the sake of brevity, I'm listing only 4 major failure points, but am very interested in hearing comments and suggestions on any issues that you believe are equally important.

Define Failure Please
Lack of adoption of a solution screams that something went wrong. It summarizes the conditions of a solution which is abandoned, isn't used as intended, does not grow and/or doesn't provide the ROI to the client as initially intended. This has nothing to do with project overrun (in terms of time and budget), but can result in project related issues if something is done too "late in the day" to remedy one of the potential BI failure points.
And when I mention the phrase lack of adoption it describes one or more of these three possible conditions:

An abandoned solution. Unused, unloved and a taboo topic within the organization.
A static solution. Cant move. Can’t adapt. It’s a short matter of time before it’s dumped.
An inappropriate solution. It does work, but in all the wrong ways, and ROI isn’t even up for consideration.

To prevent these conditions, we need to recognize the factors which cause them. The following are 4 such major factors:

  1. Lack of business buy-in
  1. Know the Answer before the Question
  1. Rubbish in, rubbish out
  1. Evolution ignored

Lack of business buy-in
Business "buy-in" refers to commitment and investment (other than economic investment) of the client in the solution. It is assumed all too often that since a client has agreed to pay for a solution, that the entire organization will instantly support and guide the effort to fruition. Not so. In fact, in many cases the solution signals change, new interfaces and access points, new concepts and new things to learn.

There are five phases here that need addressing:
  • The initial negotiations. Make it clear the amount of resource involvement that will be needed through the engagement.
  • Planning. A phased approach gives opportunity to present tangible results to the client early on. This serves two major purposes:
    1. Inspires interest in the project, and justifies buy in.
    2. Provides an early indication of the end result and gets early feedback.
  • Implementation. Have the resources feel involved, supported and benefited by helping you in your effort. If they know they have to gain from the effort, they will not only assist, but will also evangelize on your behalf.
  • User Acceptance. It is a challenge to convey the importance of this phase to the business users. On their end, they assume you have done all the heavy lifting. And true as that might be, they need to understand how critical it is that they find any flaws in your perfect solution. Motivational and leadership skills are needed here. Ironically, reverse psychology can play a part, in which as an act of revenge for taking their time, some might even try doubly hard to find issues with your solution. Use with caution.
  • Post-deployment. Once the system has been deployed, any questions and queries regarding the solution (metadata, calculations, definitions, etc.) must be quickly and easily answered from an easily accessed repository of information. In this way, you remove the obstacles for the adoption and inspire confidence in your solution.

Know the Answer before the Question
Knowing the answer before you even ask the question suggests over-confidence. Experience teaches us that even though we may believe we have the answer, the reality is sometimes quite different. In this case, such assumptions are costly, and if left until UAT phase, are impossible to rollback.
The classic example is of a client that demands a specific technical solution, despite its lack of relevant functionality. Normally as an attempt to trump the competition, or even follow the "Joneses" this rarely leads to success. The correct way to go, is to put aside the assumed "ideal solution" and see the broader scope.
One way to eliminate assumptions is to list them (some are hiding in your project-team as irrefutable beliefs), and then to flip them around as hypothetical opposites. In the case of the "ideal technical solution" assume it is the absolutely wrong solution, and then list the reasons why. If this list cannot be challenged, then perhaps it is the wrong solution.
When Thomas Edison wanted to hire a new member for his team, he would take them to lunch and order soup for them. If they seasoned the soup before tasting it, they would not be hired. Edison wanted to work with people that did not rely on assumption in their daily lives, as that would create the barrier to discovery and innovation in their work.

Rubbish in, rubbish out
Data quality is a very serious issue, and must be addressed as early as possible. Each scenario and enterprise has it's own data handling strategies, and although best-practices are well known, the world is not ideal.
Having a data handling policy (or plan of attack, call it what you will) when entering an engagement is critical.
Accuracy in the data early on, provides a solid working base from which testing of your implementation can be conducted with confidence.
Data cleansing can very well be a project on its own. The case may be that the operational systems from which you will extract data will need additional validation functionality, as well as possible data store design alterations and even procedural changes. All this would affect the data store (or stores) used by more than just the BI solution you are there to implement. As such the impact can potentially be much larger than the project you are undertaking.
  • Appoint or request a data-steward. Expert-level advice on data issues, and responsibility for the data quality.
  • Address data-quality issues immediately. In most cases these issues will take considerable time to rectify, and as such need to be incorporated into the plan promptly as dependencies.
  • The devil is in the detail. Your team must include a quality assurance stalwart, that will make audible noises at the presence of even the tiniest of data issues.
  • If possible, have data cleansing performed before the ETL processes which would feed data to the BI solution. This way the data is cleansed for the entire organization and discrepancies are eliminated.
In the event that you are drawing data from existing data-marts or a data-warehouse, the need for data cleansing is even more clearly the domain of the client. It is advisable to refrain from accepting to "fix" data as it enters your BI implementation, as this "fix" or cleansing might not be equivalent to cleansing performed by other analytics implementations using the same data. This could lead to inconsistency of information within the organization and - considering that "inconsistency" is the one thing you are aiming to remove - your efforts will be counter-productive.

Evolution ignored
Business intelligence solutions reflect the business performance on many levels, and address various business processes. Any changes to these processes should be represented in the solution, through appropriate change actions if the relevance of the BI solution is to be held intact. This will retain the relevance of the BI solution over time.
Addition of new business processes and/or their resultant data should also be considered, provided that the design is kept focused, and easy to navigate and make use of.
Major, enterprise wide changes cannot always be represented in a BI implementation without change of a relative scale, but those that can should be as part of ongoing maintenance. In short, the BI implementation should be considered an organic entity, following along with the organization.
To achieve this the following must be considered:
  • Design should be based not only on immediate requirements, but must also acknowledge possible requirements further into the future. Strong business experience would provide the best advice here, so identify business members that have expert grasp of the industry and the organizations needs into the future.
  • Service level agreements should include support and maintenance which encourage extensions and incorporation of changes into the infrastructure.
  • Review of performance should performed periodically, and gives opportunity for further co-operation and new opportunities both because of the strengthened relationship and as platform for the presentation of further infrastructure suggestions.

And final thoughts...
Obviously avoiding these points is not what will guarantee success, as success is the result of doing a great number things right (as opposed to not doing some things wrong).
The thing to take from this article is the opportunity to not spoil a great project by ignoring these points, as they are quite often neglected and most often result in a tragically spoiled effort.

No comments: