Your Company’s Problem with Data may be Fear

I just read this article by Ethan Knox, “Is Your Company Too Dumb to be Data Driven” and was intrigued to read what many people have discussed for years.  I’ve spent nearly half my career helping clients make the transition from running the business by tribal knowledge and gut instinct to running the business by facts and numbers.  It’s a hard transition.  One that takes vision, motivation, discipline, and courage to change.  It also takes a willingness to learn something new.     

While this article covers a lot of ground, I wanted to comment on one of points made in the article: the mistake of “build it and they will come”.  This occurs when an organization is enthusiastic about data and decides to build a data warehouse (or data lake) and load it with all the data from the company’s core application systems (sales, finance, operations, etc.)  The whole business case depends on the users flocking to the system, using new business intelligence or reporting tools, and uncovering numerous high value business insights.  All too often, the results reflect a large monolithic data platform that contains lots of content but hasn’t been designed to support analysis or decision making by the masses.

There are numerous problems with this approach – and the path to data and analytics enlightenment is littered with mistakes where companies took this approach.   Don’t assume that successful companies that have embraced data and analytics didn’t make this mistake (it’s a very common mistake).  Successful companies were those that were willing to learn from their mistakes – and have a culture where new project efforts are carefully scoped to allow mistakes, learning, and evolution.   It’s not that they’re brilliant; successful companies understand that transitioning to being data driven company requires building knowledge.  And, the process of learning takes time, includes mistakes, requires self-analysis, and must be managed and mentored carefully.  They design their projects assuming mistakes and surprises occur, so they fail fast and demand continual measurement and corrective action.  It’s not about the methodology or development approach.  A fail-fast philosophy can work with any type of development methodology (agile, iterative, waterfall). The path to data enlightenment will include lots of mistakes.

Do you remember high school math?  When you were presented with a new concept, you were given homework that allowed you to learn, gain experience, and understand the concept through the act of “doing”.  Homework was often graded based on effort, not accuracy (if you did it, you got credit, whether or not it was correct).  Where is it written that (upon graduation) learning something new wouldn’t require the act of “doing” and making mistakes to gain enlightenment?  By the way, who has ever succeeded without making mistakes?

The point the article frequently references it that business engagement is critical.  It’s not about the users participating a few times (requirements gathering and user acceptance testing); it’s about users being engaged to review results and participate in the measurement and corrective action.  It’s about evolving from a culture where the relationship is customer/ provider to a team where everyone succeeds or fails based on business measurement.

It’s not that a company is too dumb to succeed with data; it’s that they’re often too fearful of mistakes to succeed.  And in the world of imperfect data, exploding data volumes, frequent technology changes, and a competitive business environment, mistakes are an indication of learning.   Failure isn’t a reflection of mistakes, it’s a reflection of poor planning, lack of measurement, and an inability to take corrective action.