Good Stats Make Us Uncomfortable (IO Psychology)
In striving for profitability, companies often rely on key indicators of organizational performance. Common indicators like sales growth, customer loyalty, and earnings per share often guide strategy decisions and resource allocation. But sometimes key indicators may not be that “key” after all. They may have little or no true connection to profitability.
Organizations might not be aware of this and continue to rely on these same measures because they feel as though they matter. Their intuition overrides everything else and as a result they don’t do the due diligence to determine what actually leads to profit. They become overconfident, grab onto any numbers that are easily available, and rely on things they have always looked at in the past. They choose what they like and what feels comfortable. But they don’t actually analyze. So how can we know if something truly predicts value? We cannot leverage something we don’t know.
This article focuses on identifying indicators that serve as true statistical predictors of value. The author emphasizes that for an indicator to be truly connected to value it must be both predictive and persistent. Indicators that are predictive demonstrate a statistical link to value; a link strong enough that we feel confident saying there is a connection that has meaning and is not due to chance. Indicators that are persistent stand the test of time; they reliably show that that an outcome is controlled by applying skill or knowledge, and is not random.
The author advocates several steps in selecting the best indicators of organizational performance. These steps include defining a clear business objective, developing theories to determine what measures might link to the objective, and statistically testing the relationship between the measures and the objective.
As I read these steps they made complete sense to me, but my data-happy left brain went nuts, thinking about others questions that should be considered. Like: “What else do we already measure?…Could it matter?” and “What else can we measure?…What else should we measure?” and “What other viewpoints are we not taking into account?” and “What curveballs could come our way?”
Sometimes a meaningful statistic can push us out of our comfort zone. Actually, sometimes a meaningful statistic should push us out of our comfort zone. It might not make automatic, inherent sense to us, especially at first. If all statistics made complete, gut-happy sense to us we wouldn’t need them. We could always rely on our intuition because it would always be correct. But statistics are useful because they not only tell us how meaningful things might be related; they can surprise us with the sheer fact of what things might be related.
If a predictor of success isn’t pointing in the direction of success, it’s not a predictor. It’s simply a number. And a useless one at that.