The Real Key to Great Analytics?
By: Jeff Headley
When it comes to data analytics, the process looks something similar to this:
Define the problem
Decide how to measure the outcome
Collect data
Analyze
Interpret the results
For me, one of these steps outweighs the others by a wide margin. What’s your vote?
Defining the problem is the single most important step in all of analytics.
That’s not to say the other steps aren’t important. They very much are. But failing to properly scope what you’re trying to solve, and most importantly, why, leads to a lot of wasted effort and frustration. And if this becomes too common of an occurrence, analytics within your company may start to lose its luster.
At one of my previous analytics firms, we used to have a saying when a challenge was posed by a client, “What’s the question behind the question?” Simply translated, people rarely accurately and succinctly articulate their real question or problem. Getting a proper handle on an analysis project usually takes a combination of deconstructing the challenge into its component parts, thoughtfulness, and a whole lot of questions.
If you’re a parent or have otherwise spent a lot of time with young children, there is a 100% chance you’ve been bombarded with a never ending stream of questions….each one building on the previous. Research shows that the more questions a child asks, the more the child is learning.
Learning also needs to occur in the early phase of an analytics project. And it may sometimes feel like asking too many questions is a nuisance for the client or demonstrates a lack of understanding. But asking the right questions in the right way is crucial for building a sound foundation on which to build your analytics. All too often the right questions don’t get asked.
Why don’t the right questions get asked?
Everyone already knows the answer
Some questions may seem so obvious, why ask them? Making assumptions without confirming them is always dangerous.
Pre-defining the problem
There can be a strong, natural tendency to categorize a problem as soon as you hear it. There’s nothing wrong with that, but don’t let your early instincts distract you from exploring multiple angles.
Preoccupation with later analysis steps
Many analytics projects seem to skimp on understanding the problem and instead get caught up in things like “do we have the right data”, “what analysis technique should we be using”, etc.
Start with the end in mind
There are many different ways an analysis team can coax the right detail out of their stakeholders. Here is one method that consistently works for me:
In this thought experiment, your goal is to get your audience to think “physically” about the insight they want to leverage. Ask them to imagine that the insight was magically brought into existence. Now get them to explain how this reality will operate.
How does the insight get presented? I.e. report, dashboard, API
Does this insight have a “home” in an existing system or will something new need to be created to house it?
Who is interacting with the insight? How often?
What decisions are being made or influenced?
Who or what team with the organization will ensure the insight is being applied as intended?
What KPIs or measurements will be used to gauge performance?
I find that this initial focus on the practical, operational aspects of a proposed analytics project is very helpful in gauging how deeply your stakeholders have thought about their request, as well as the level to which their expectations are aligned.
Then, it’s time for more questions….