So, What Is Your Algorithm?

I thought this story about Schwan’s was interesting for this reason: a 3-4% improvement on revenues of ~$3 billion (2010 Annual report) over less than one year didn’t feel that significant to me.  In fact, if you look at it this way, Schwan’s improved sales by 3.5% * $3billion/3million purchasers = $35/purchaser.  That’s two additional entrees, or 5 additional pizzas, per customer over 1 year!

The 3-4% improvement over 1 year doesn’t mean, either, that Schwan’s will maintain annual revenue growth of 3-4%, especially if their customer base doesn’t grow.  In fact, according to their 2011 Annual Report, revenues remained at the ~$3 billion level as 2010. So what I see here, in the limited amount of information in this story, is that Schwan’s system lifted sales to make their fleet incrementally more efficient.

Average inflation for 2011 was reported to be 3.2%. (See "Table of Inflation Rates by Month and Year (1999-2012)")  I would be interested to know if Schwan’s own expenses grew at the inflation rate.  If so, Schwan’s merely kept up with inflation through 2011.

Depending on the cost of the system, the system may have made sense from a stand alone ROI perspective.  Call me skeptical, though, but this doesn’t feel like a sustainable game changing improvement at Schwan’s. I hope the story is different for Schwan's one year down the road from the original publication date of this article.

All that aside, I’m not sure this is a good example of avoiding the kinds of decision/thinking failures that Daniel Kahneman talked about because it seems to me the Schwan’s fleet is simply getting a little bit better information about how to implement an existing strategy, as opposed to Schwan’s avoiding the kinds of biases that make people pick the wrong strategy. Maybe that’s the real story here – Schwan’s let the siren song of advanced technology convince them to continue following a margin sensitive strategy to 1 (!) more decimal place.  They are solving the wrong problem with increasing precision. In fact, this story about Schwan’s isn’t really consistent with the Moneyball story of Billy Beane and the Oakland A’s.  In that example, Billy Beane and Paul Podesta challenged long held beliefs about the value of players’ capabilities, tested their own hypotheses, and bought the resources they needed to win at bargain rates.  They weren’t just squeezing more runs out of superstar players.  They found value where everyone else who esteemed themselves as experts said that it couldn’t be found.  The effect was actually, pardon the pun, game changing for the A’s.  They were no longer building a baseball team or playing baseball the way everyone else said that it had to be managed and played.  They became a uniquely good team versus being a team that tried only to improve within notions of conventional wisdom with declining marginal returns for the effort.  Did they use statistical analysis to do their job? Yes, but I think they really only needed the statistical analysis to indicate the presence of an inefficiency in the baseball marketplace, and then to find the resources they needed. They formulated and pursued a strategy around this idea of exploiting information inefficiencies.

What does this mean for us —those of us who desire to be better at making more valuable and creative decisions or helping others in that endeavor?  First, in the age of Big Data, I think we need to be careful about showcasing Big Data applications as examples of how decision analysis methodologies work.  The story about Schwan’s in this article is an application of data analytics and information technology that gleaned narrow improvements from statistical information, not necessarily good creative decision making.  We need to be careful about the distinction in what some people are calling applications of decision science and what we do with decision analysis and management.  I don’t doubt that the Opera solution is doing advanced analytics. I just doubt that Schwan’s engaged in good decision making. :/

What I see that valuable decision analysis provides is a mindset, a meta-system, to avoid the kind of system 1 (biased intuition) and system 2 failures (intellectual laziness) that Kahneman describes. A thorough application of decision analysis should consider multiple alternate strategies to achieve something more than incremental improvements.  With decision analysis we should seek disconfirming evidence and logic for biased assumptions.  Decision analysis of the kind I think we want to do avoids cognitive inefficiencies that arise from cognitive and motivational biases, information that has been aggregated at too gross of a level, and creative laziness. The questions we ought to help the consumers of our thinking answer are not just whether they need better information systems, but whether, for example, a better information system is the best application of resources to achieve game changing returns.  I think this kind of thinking leads to qualitatively different kinds of question.  I’m not saying that advanced data analysis can’t be powerful. We know that it can be.  But it might not always be the best solution.