While many medical professionals spend their days juggling patients and insurance companies (and more!), they are also expected to keep up with the latest research. Research that might save lives and improve quality of lives are obviously important, but what about the integration of new or novel data?
Change in the medical world takes years. Possibly decades. This passage from Atul Gawande’s The Checklist Manifesto speaks to the difficulty and delay in adoption of new or novel data derived from research:
Sometimes, though, failures are investigated. We learn better ways of doing things. And then what happens? Well, the findings might turn up in a course or seminar, or they might make it into a professional journal or a textbook. In ideal circumstances, we issue some inch-thick set of guidelines or a declaration of standards. But getting the word out is far from assured, and incorporating the changes often takes years.
One Study in medicine, for example, examined the aftermath of nine different major treatment discoveries such as the finding that the pneumococus vaccine protects not only children but also adults from respiratory infections, one of our most common killers. On average, the study reported, it took doctors SEVENTEEN YEARS to adopt the new treatments for at least half of American patients.
What experts like Dan Boorman have recognized is that the reason for the delay is not usually laziness or unwillingness. The reason is more often that the necessary knowledge has not been translated into a simple, usable, and systematic form. If the only thing people did in aviation was issue dense, pages-long bulletins for every new finding that might affect the safe operation of airplanes – well, it would be like subjecting pilots to the same data deluge of almost 700,000 medical journal articles per year that clinicians must contend with. The information would be unmanageable.
While we cannot deny the importance of medical research, it is just as (maybe even more) important that the information is structured into an actionable architecture for efficient practicality.
Decision-making isn’t always the easiest thing in the world. While many errors may seem obvious in hindsight, they’re rarely as crystal clear during the decision-making process. Even worse, we have a tough time imagining the opposing view. As Michael Mauboussin states in his book Think Twice, we have “a tendency to favor the inside view over the outside view.” He goes on to explain,
An inside view considers a problem by focusing on the specific task and by using information that is close at hand, and makes predictions based on that narrow and uniques set of inputs. These inputs may include anecdotal evidence and fallacious perceptions. This is the approach that most people use in building models of the future and is indeed common for all forms of planning.
Compare that with The Outside View:
The outside view asks if there are similar situations that can provide a statistical basis for making a decision. Rather than seeing a problem as unique, the outside view wants to know if others have faced comparable problems and, if so, what happened. The outside view is an unnatural way to think, precisely because it forces people to set aside all the cherished information they have gathered…. The outside view can often create a very valuable reality check for decision makers.
He goes on to list three illusions that lead one to the inside view: the Illusion of Superiority (I’m better than them), the Illusion of Optimism (that’ll never happen to me), and the Illusion of Control (I can make this happen). Obvious question: “How can we get better at adopting The Outside View?”
Mauboussin pulls from Kahneman and Tversky, and distills their 5 step process into 4 steps.
- Select a reference class: “Find a group of situations, or a reference class, that is broad enough to be statistically significant but narrow enough to be useful in analyzing the decision that you face. The task is generally as much art as science, and is certainly trickier for problems that few people have dealt with before. But for decisions that are common – even if they are not common for you – identifying a reference class is straightforward.”
- Assess the distribution of outcomes: “Once you have a reference class, take a close look at the rate of success and failure…. Two other issues worth mentioning. The statistical rate of success and failure must be reasonably stable over time for a reference class to be valid. If the properties of the system change, drawing inference from past data can be misleading…. Also keep an eye out for systems where small perturbations can lead to large-scale change. Since cause and effect are difficult to pin down in these systems, drawing on past experiences is more difficult.”
- Make a prediction: “With the data from your reference class in had, including an awareness of the distribution of outcomes, you are in a position to make a forecast. The idea is to estimate you chances of success and failure…. Sometimes when you find the right reference class, you can see the success rate is not very high. So to improve your chance of success, you have to do something different that everyone else.”
- Assess the reliability of your prediction and fine-tune: “How good we are at making decisions depends a great deal on what we are trying to predict. Weather forecasters, for instance, do a pretty good job of predicting what the temperature will be tomorrow. Book publishers, on the other hand, are poor at picking winners, with the exception of those books from a handful of best-selling authors. The worse the record of successful prediction is, the more you should adjust your prediction toward the mean (or other relevant statistical measure). When cause and effect is clear, you can have more confidence in your forecast.”
The more probabilistic the context, the better these step will work. Now you know how to take The Outside View to increase the odds of a better decision.
The main lesson from the inside-outside view is that while decision makers tend to dwell on uniqueness, the best decisions often derive from sameness. – Mauboussin