Showing posts with label decision analysis. Show all posts
Showing posts with label decision analysis. Show all posts

Wednesday, November 09, 2016

The Power of Negative Thinking: “How do we know this opportunity is worth the time and effort?”

The sales process is an inherently risky business. It’s difficult to know if and when a deal will close, what clients really want regardless of what they have stated (i.e., the client may have failed to frame their own needs properly), and what competitors offer in price and quality of deliverables.

Compounding the external uncertainty, we often get in our own way by importing certain kinds of biases into our assessment of the value of the sales opportunities at hand. These biases can include…
  • Unwarranted optimism or wishful thinking – personal enthusiasm or a natural disposition to believe that desired outcomes will most likely occur; or, inflating initial estimates of desired outcomes to appear more effective than is warranted;
  • Sand-bagging – under reporting potential outcomes to appear heroic when better than anticipated outcomes materialize;
  • False precision – reporting anticipated outcomes with an unjustified level of certainty, usually as a single-point estimate rather than a range;
  • Availability – recalling values that are memorable, easily accessible, recent, or extreme;
  • Anchoring – using the first “best guess” as a starting point for subsequent estimating;
  • Expert over-confidence – failure of creativity or hubris (e.g., “I know this information and can’t be wrong because I’m the expert.”);
  • Incentives – the SME experiences some benefit or cost in relationship to the outcome of the term being measured, adjusting his estimate in the direction of the preferred outcome;
  • Entitlement – the SME provides an estimate that reinforces his sense of personal value.
Without bias-free assessments in our decisions to actively pursue sales opportunities, it's nearly impossible to know how to allocate sales and support resources effectively to maximize the likelihood of capturing sales in a profitable and efficient manner. In short, when given the opportunity to pursue multiple opportunities with limited resources, it’s often difficult to know if any given opportunity is worth the time.

As odd as it may sound in a culture that seems to demand almost endless optimism, the Power of Negative Thinking actually helps us to overcome our biases as well as inform us how to obtain better information about the external uncertainties we face. By “negative thinking” we do not mean cynicism or toxic nay-saying. Rather, we refer to a process that asks us to consider critically the opposite of what we too easily assume (or wish) to be true. While Negative Thinking could lead us to consider the effects of unfortunate outcomes or conditions (the opposite of desired outcomes) on sales opportunities...

The best laid schemes o’ Mice an’ Salesmen, Gang aft agley


...it could also lead us to consider the possibility of desirable outcomes or conditions (the opposite of the unfortunate) for situations that we often easily dismiss.

No, no, boy, that's no way to make a plane. That'll, I say, that'll never...fly!

But the Power of Negative Thinking goes beyond our merely considering what can happen. We must also consider the “why” and “to what degree” those things could happen. We can account for the “what,” “why,” and “to what degree” in a process called probabilistic reasoning. But that's the second step. The Power of Negative Thinking begins with accurately framing an opportunity, which requires that a sales team answer the following questions:
  • What is the real opportunity? 
  • What are our goals and objectives?
  • What are the client's goals and objectives?
  • What are the decision boundaries and open decisions?
  • What are the sources of uncertainty? 
Answering these questions helps the team know that it has the right reasons in mind to pursue an opportunity and what constraints in their current level of knowledge limit their ability to make unambiguous decisions about what opportunities to pursue and how to go about pursuing them.

Probabilistic reasoning helps a sales team then answer these questions:
  • What is the likely range of outcomes for the uncertainties? 
  • What are the effects of uncertainties on sales goals, revenues, and profit? 
  • How much risk do we face with each opportunity; i.e., how much could we lose by pursuing one opportunity over another?
  • What insights can we create for contingency plans or options?
  • How do we prioritize our set of current opportunities?

The effect of taking these two steps in a structured way reveals the Power of Negative Thinking so that the sales team can recognize when an opportunity is worth pursuing…or not. Ultimately, not only does the Power of Negative Thinking give the sales team a more accurate assessment of the current state and possibilities they face, they can also develop more effective contingency plans to increase the likelihood of achieving results their organization—and their clients—desire.

A Decision Analyst's View of Electoral Surprise

I turned off the television last night at 8 PM. Since I had an analytics problem to work on, I didn't want my attention divided, and I knew that clinging to electoral results was more neurotic than helpful. My attention at the moment was not going to change the results. So, I rolled up my sleeves and got to work.

At 12:30 AM, I turned my television back on...


As I watched the polling results roll in and followed the reactions of establishment pundits and the broader hoi polloi (from both sides) in social media, all I could think was, "What is going on here?" Over and over. I mean, Nate Silver was still giving better than 2:1 odds of a Clinton victory just before I turned off the TV. Could the situation really have been that different than assessed? Could things really have changed that quickly? At 4 AM, I finally captured some thoughts that I think should serve as object lessons for all of us, and not just in politics, but in business, too.
  1. Never, ever believe your own spin. Humans love narratives that give them comfort. Unfortunately, almost all narratives are constructed from selected evidence that fits a preferred narrative.
  2. Always question where your biases are coming from. You are biased. Until you recognize it, you will frequently be rudely embarrassed. 
  3. There is no meaningful position in certainty. All beliefs about future events should be treated with degrees of belief. 
  4. Even events that happened in the past are open to interpretation. The real issue about the facts of events is not so much whether events have occurred in the past or whether they will occur in the future. The real issue is our epistemic distance from the events. We generally don't know as much as we think we do.
  5. We condition our beliefs on the evidence at hand. Thinking that a Clinton victory was highly probable was not a bad position to take. It made sense given much of the evidence. BUT, Prob(Clinton win) > 50% does mean Prob(Clinton win) = 100%! (I'm actually getting tired of explaining this. I'm getting tired of seeing people make this mistake and the effects it has in real life on real people. Probabilities are degrees of belief, not statements of fact.) Always, always, always consider the disconfirming evidence. 
  6. Trump never showed an insignificant chance of winning. His victory was always probable. What I see and hear coming from those expressing shocked disappointment about the Clinton loss is that they didn't really explore and consider the edge cases that would lead to a Trump victory. Explore the edge cases. Explore aggressively. Keep exploring. 
  7. Informed accuracy trumps false precision (pun intended). Don't be embarrassed to draw your prediction intervals wide. It's more honest, more informative, and will allow you to do a better job preparing contingency plans. When #6 is performed honestly and aggressively, it should lead you to make your prediction intervals even wider. It's better to be humble and recognize how little you know versus being sure and then being rudely surprised.
  8. The evolving probability of win curves for this election resemble the curves associated with predicting that a given hypothesis among several is true when there are unaccounted for characteristics at play. Suddenly, a seemingly most likely explanation crashes to be replaced by a previously less likely hypothesis as the unrecognized characteristic manifests itself. This is a long way to say people get caught up in false dichotomies (or n-chotomies) for the possible explanations for what really is the case. It is almost always the case that more explanations are available than the limited set we originally conceived.
  9. If something really weird happens and somehow the posted results at 4 AM reverse by the time I wake up, all of the above still applies, maybe more so.

Although Nate Silver was leaning in the wrong direction for predicting the outcome, his odds were actually more realistic and informed than many other pollsters who were giving 19:1 odds or better for a Clinton win.

Tuesday, September 10, 2013

It's Your Move: Creating Valuable Decision Options When You Don't Know What to Do

The followings is the first chapter excerpt from my newly published tutorial.

Business opportunities of moderate to even light complexity often expose decision makers to hundreds, if not tens of thousands, of coordinated decision options that should be considered thoughtfully before making resource commitments. That complexity is just overwhelming! Unfortunately, the typical response is either analysis paralysis or "shooting from the hip," both of which expose decision makers to unnecessary loss of value and risk. This tutorial teaches decision makers how to tame option complexity to develop creative, valuable decision strategies that range from "mild to wild" with three simple thinking tools.


Read more here.

Tuesday, July 23, 2013

Business Case Analysis with R

The following is the first chapter excerpt from my newly published book.

Business Case Analysis with R

A Simulation Tutorial to Support Complex Business Decisions


1.2 Why use R for Business Case Analysis?
Even if you are new to R, you most likely have noticed that R is used almost exclusively for statistical analysis, as it's described at The R Project for Statistical Computing. Most people who use R do not frequently employ it for the type of inquiry which business case analysts use spreadsheets to select projects to implement, make capital allocation decisions, or justify strategic pursuits. The statistical analysis from R might inform those decisions, but most business case analysts don't employ R for those types of activities.

Obviously, as the title of this document suggests, I am recommending a different approach from the status quo. I'm not just suggesting that R might be a useful replacement for spreadsheets; rather, I'm suggesting that better alternatives to spreadsheets be found for doing business case analysis. I think R is a great candidate. Before I explain why, let me explain why I don't like spreadsheets.

Think about how a spreadsheet communicates information. It essentially uses three layers of presentation:
  1. Tabulation
  2. Formulation
  3. Logic
When we open a spreadsheet, usually the first thing we see are tables and tables of numbers. The tables may have explanatory column and row headers. The cells may have descriptive comments inserted to provide some deeper explanation. Failure to provide these explanatory clues represents more a failing of the spreadsheet developer's communication abilities than a failing of the spreadsheet environment, but even with the best of explanations, the emergent pattern implied by the values in the cells can be difficult to discern. Fortunately, spreadsheet developers can supply graphs of the results, but even those can be misleading chart junk.

To understand how the numbers arise, we might ask about the formulas. By clicking in a cell we can see the formulas used, but unfortunately the situation here is even worse than the prior level of presentation of tables of featureless numbers. Here, we don't see formulas written in a form that reveals underlying meaning; rather, we see formulas constructed by pointing to other cell locations on the sheet. Spreadsheet formulation is inherently tied to the structural presentation of the spreadsheet. This is like saying the meaning of our lives should be dependent on the placement of furniture in our houses.

While the goal of good analysis should not be more complex models, a deeper inquiry into a subject usually does create a need for some level of complexity that exceeds the simplistic. But as a spreadsheet grows in complexity, it becomes increasingly difficult to extend the size of tables (both by length of indices that structure them and the number of indicies used to configure the dimensionality) as a direct function of its current configuration. Furthermore, if we need to add new tables, choosing where to place them and how to configure them also depends almost entirely on the placement and configuration of previously constructed tables. So, as the complexity of a spreadsheet does increase, it naturally leads to less flexibility in the way the model can be represented. It becomes crystalized by the development of its own real estate.

The cell referencing formulation method also increases the likelihood of error propagation because formulas are generally written in a quasi-fractal manner that requires the formula to be written across every element in at least one index of a table's organizing structure. Usually, the first instance of a required formula is written within one element in the table; then, it is copied to all the appropriate adjacent cells. If the first formula is incorrect, all the copies will be, too. If the formula is sufficiently long and complex, reading it to properly debug it becomes very difficult. Really, the formula doesn't have to be that complicated or the model that complex for this kind of failure to occur, as the recent London Whale VaR model and Reinhart-Rogoff Study On Debt debacles demonstrated.[1]

All of this builds to the most important failure of spreadsheets -- the failure to clearly communicate the underlying meaning and logic of the analytic model. The first layer visually presents the numbers, but the patterns in them are difficult to discern unless good graphical representations are employed. The second layer, which is only visible unless requested, uses an arcane formulation language that seems inherently irrational compared to the goal of good analysis. The final layer--the logic, the meaning, the essence of the model--is left almost entirely to the inference capability of any user, other than the developer, who happens to need to use the model. The most important layer is the most ambiguous, the least obvious. I think the order should be the exact opposite.

When I bring up these complaints, the first response I usually get is: "ROB! Can't we just eat our dinner without you complaining about spreadsheets again?" But when the population of my dinner company tends to look more like fellow analysts, I get, "So what? Spreadsheets are cheap and ubiquitous. Everyone has one, and just about anyone can figure out how to put numbers in them. I can give my analysis to anyone, and anyone can open it up and read it."

Then I'm logically--no, morally--compelled to point out that carbon monoxide is cheap and ubiquitous, that everyone has secrets, that just about everyone knows how to contribute to the sewage system, that just about everyone can read your diary and add something to it. Free, ubiquitous, and easy to use are all great characteristics of some things in their proper context, but they aren't characteristics that are necessarily universally beneficial.

More seriously, though, I know that what most people have in mind with the common response I receive is the low cost of entry to the use of spreadsheets and the relative ease of use for creating reports (which I think spreadsheets are excellent for, by the way). Considering the shortcomings and failure of spreadsheets based on the persistent errors I've seen in client spreadsheets and the humiliating ones I've created, I think the price of cheap is too high. The answer to the first part of their objection--spreadsheets are cheap--is that R is free. Freer, in fact, than spreadsheets. In some sense, it's even easier to use since the formulation layer can be written directly in a simple text file without intermediate development environments. Of course, R is not ubiquitous, but it is freely available on the internet.

Unlike spreadsheets, R is programming language with the built in capacity to operate over arrays as if they were whole objects, a feature that demolishes any justification for cell-referencing syntax of spreadsheets. Consider the following example.

Suppose we want to model a simple parabola over the interval (-10, 10). In R, we might start by defining an index we call x.axis as an integer series.

x.axis <– -10:10

which looks like this,

[1] -10  -9  -8  -7  -6  -5  -4  -3  -2  -1 0  1  2  3  4  5  6  7  8  9  10

when we call x.axis.

To define a simple parabola, we then write a formula that we might define as

parabola <– x.axis^2

which produces, as you might now expect, a series that looks like this:

>[1] 100  81  64  49  36  25  16  9  4  1  0  1  4  9  16  25  36  49  64  81 100.

Producing this result in R required exactly two formulas. A typical spreadsheet that replicates this same example requires manually typing in 21 numbers and then 21 formulas, each pointing to the particular value in the series we represented with x.axis. The spreadsheet version produces 42 opportunities for error. Even if we use a formula to create the spreadsheet analog of the x.axis values, the number of opportunities for failure remains the same.

Extending the range of parabola requires little more than changing the parameters in the x.axis definition. No additional formulas need be written, which is not the case if we needed to extend the same calculation in our spreadsheet. There, more formulas need to be written, and the number of potential opportunities for error continues to increase.

The number of formula errors that are possible in R is directly related to the total number of formula parameters required to correctly write each formula. In a spreadsheet, the number of formula errors is a function of both the number of formula parameters and the number of cell locations needed to represent the full response range of results. Can we make errors in R-based analysis? Of course, but the potential for those errors is exponentially smaller.

As we've already seen, too, R operates according to a linear flow that guides the development of logic. Also, variables can be named in a way that makes sense to the context of the problem[2] so that the program formulation and business logic are more closely merged, reducing the burden of inference about the meaning of formulas for auditors and other users. In Chapter 2, I'll present a style guide that will help you maintain clarity in the definition of variables, function, and files.

However, while R answers the concerns of direct cost and the propagation of formula errors, its procedural language structure presents a higher barrier to improper use because it requires a more rational, structured logic than is required by spreadsheets, requiring a rigor that people usually learn from programming and software design. The best aspect of R is that it communicates the formulation and logic layer of an analysis in a more straightforward manner as the procedural instructions for performing calculations. It preserves the flow of thought that is necessary to move from starting assumptions to conclusions. The numerical layer is presented only when requested, but logic and formulation are more visibly available. As we move forward through this tutorial, I'll explain more how these features present themselves for effective business case analysis.

1.3 What You Will Learn
This document is a tutorial for learning how to use the statistical programming language R to develop a business case simulation and analysis. I assume you possess at least the skill level of a novice R user.

The tutorial will consider the case in which a chemical manufacturing company considers constructing a new chemical reactor and production facility to bring a new compound to market. There are several uncertainties and risks involved, including the possibility that a competitor brings a similar product online. The company must determine the value of making the decision to move forward and where they might prioritize their attention to make a more informed and robust decision.

The purpose of the book is not to teach you R in a broad manner. There are plenty of resources that do that well now. Rather, it will attempt to show you how to

  • Set up a business case abstraction for clear communication of the analysis
  • Model the inherent uncertainties and resultant risks in the problem with Monte Carlo simulation
  • Communicate the results graphically
  • Draw appropriate insights from the results
So, while you will not necessarily become a power user of R, you will gain some insights into how to use this powerful language to escape the foolish consistency of spreadsheet dependency. There is a better way.

1.4 What You Will Need
To follow this tutorial, you will need to download and install the latest version of R for your particular OS. R can be obtained here. Since I wrote this tutorial with the near beginner in mind, you will only need the base install of R and no additional packages.


Notes
1: You will find other examples of spreadsheet errors at Raymond Panko's website. Panko researches the cause and prevalence of spreadsheet errors.

2: Spreadsheets allow the use of named references, but the naming convention can become unwieldy if sections in an array need different names.


Read more here: Or, if you prefer Amazon or Scribd.