Two experts provide a forecast you are interested in. How do you best use these two forecasts?
Thought experiment: Guessing the number of coins in a jar, suppose the correct answer is 40.
Two heads are better than one! Even the same head over time is better than just once
This is especially true when the two heads use different data, backgrounds and approaches.
Research by Jack Soll and Rick Larrick (Duke University) suggests, though
•People do not seem to believe in averaging
•They believe, incorrectly, that Averaging = Average performance
•Overconfident in their ability to “spot the expert”
Scenario1: You could take advice from two people who have access to the same data:
1)Mr. A, who has the same background and training as you. 2)Mr. B, whose background and training is different.
Scenario2: You could take advice from two people who have the same background as you:
1)Ms. X, who has access to the same data as you. 2)Ms. Y, who has access to a different dataset.
Many people choose “A” and “X” respectively.
However B and Y might be better choices.
2)Case Based (also called Database)
Hoch S and D Schkade (1996), Management Science
Provides the user with a score based on certain attributes or cues.
Credit Rating = a + b1 (Debt Ratio) + b2 (Cash Flow) + b3 (Revenue Trend) + b4 (Location)
The coefficients could come from
a) expert judgments,
b) a judgment bootstrap, of
c) a regression based on past data
•The DSS retrieves past instances that are most similar to the current one using a “euclidian distance” algorithm.
•The user than then employ an anchoring and adjustment process to make a judgment or prediction.
•A sample screen might look like this:
•In stable environments, model based DSS helped more than case based
•In noisy environments, case DSS was slightly better
•In either case, a combination of both is the best
•Blattberg R and S Hoch (1990), Management Science conducted several experiments comparing the performance of managerial intuition with a model’s prediction.
•Performance = Correlation between the prediction and the actual outcome
•Why models are better
–Models are consistent and mathematically correct
–They are unbiased and unaffected by framing, politics and emotions
–They do not get tired
•Why expert intuition is better
–Models only know what the expert tells it
–Experts can value qualitative cues well
–Experts can change decision-making strategy as a function of environment
–Experts have access to more cues
A to F represent six different studies. The height of the bars represent performance.
In all cases, an equally weighted average outperformed both the model and the expert.
•Let the expert and the model make independent judgments
•Take an equally weighted average
•Food for thought: If you rely on decision support, how can you best incorporate the decision-maker’s intuition or judgment?