The effects of presenting imprecise probabilities in intelligence forecasts

Nathan Dieckmann, Robert Mauro, Paul Slovic

Research output: Contribution to journalArticle

22 Citations (Scopus)

Abstract

How to assess and present analytic uncertainty to policymakers has emerged as an important topic in risk and policy analysis. Due to the complexity and deep uncertainty present in many forecasting domains, these reports are often fraught with analytic uncertainty. In three studies, we explore the effect of presenting probability assessments and analytic uncertainty through probability ranges. Participants were presented with mock intelligence forecasts that include narrative evidence as well as numerical probability assessments. Participants were sensitive to the ambiguity communicated through the confidence range. The narrative appeared to have a smaller effect on judgments when accompanied by a probability range as opposed to a point assessment. In one study, participants also thought that the probability range was more useful for decision making at a higher probability whereas the point estimate was more useful at a lower probability. When evaluating a forecast in hindsight, decisionmakers tended to report lower levels of blame and higher levels of source credibility for forecasts that reported ranges as compared to point assessments. These findings suggest that decisionmakers are not necessarily "ambiguity averse" in the forecasting context. Presenting ranges of probability may have distinct advantages as a way to communicate probability and analytic confidence to decisionmakers.

Original languageEnglish (US)
Pages (from-to)987-1001
Number of pages15
JournalRisk Analysis
Volume30
Issue number6
DOIs
StatePublished - Jun 2010
Externally publishedYes

Fingerprint

Intelligence
Uncertainty
Policy Making
Decision Making
Decision making

Keywords

  • Ambiguity
  • Analytic uncertainty
  • Forecasting
  • Imprecise probability
  • Risk communication

ASJC Scopus subject areas

  • Physiology (medical)
  • Safety, Risk, Reliability and Quality
  • Medicine(all)

Cite this

The effects of presenting imprecise probabilities in intelligence forecasts. / Dieckmann, Nathan; Mauro, Robert; Slovic, Paul.

In: Risk Analysis, Vol. 30, No. 6, 06.2010, p. 987-1001.

Research output: Contribution to journalArticle

Dieckmann, Nathan ; Mauro, Robert ; Slovic, Paul. / The effects of presenting imprecise probabilities in intelligence forecasts. In: Risk Analysis. 2010 ; Vol. 30, No. 6. pp. 987-1001.
@article{ba7443f67d284fd0aa947258ea2eec61,
title = "The effects of presenting imprecise probabilities in intelligence forecasts",
abstract = "How to assess and present analytic uncertainty to policymakers has emerged as an important topic in risk and policy analysis. Due to the complexity and deep uncertainty present in many forecasting domains, these reports are often fraught with analytic uncertainty. In three studies, we explore the effect of presenting probability assessments and analytic uncertainty through probability ranges. Participants were presented with mock intelligence forecasts that include narrative evidence as well as numerical probability assessments. Participants were sensitive to the ambiguity communicated through the confidence range. The narrative appeared to have a smaller effect on judgments when accompanied by a probability range as opposed to a point assessment. In one study, participants also thought that the probability range was more useful for decision making at a higher probability whereas the point estimate was more useful at a lower probability. When evaluating a forecast in hindsight, decisionmakers tended to report lower levels of blame and higher levels of source credibility for forecasts that reported ranges as compared to point assessments. These findings suggest that decisionmakers are not necessarily {"}ambiguity averse{"} in the forecasting context. Presenting ranges of probability may have distinct advantages as a way to communicate probability and analytic confidence to decisionmakers.",
keywords = "Ambiguity, Analytic uncertainty, Forecasting, Imprecise probability, Risk communication",
author = "Nathan Dieckmann and Robert Mauro and Paul Slovic",
year = "2010",
month = "6",
doi = "10.1111/j.1539-6924.2010.01384.x",
language = "English (US)",
volume = "30",
pages = "987--1001",
journal = "Risk Analysis",
issn = "0272-4332",
publisher = "Wiley-Blackwell",
number = "6",

}

TY - JOUR

T1 - The effects of presenting imprecise probabilities in intelligence forecasts

AU - Dieckmann, Nathan

AU - Mauro, Robert

AU - Slovic, Paul

PY - 2010/6

Y1 - 2010/6

N2 - How to assess and present analytic uncertainty to policymakers has emerged as an important topic in risk and policy analysis. Due to the complexity and deep uncertainty present in many forecasting domains, these reports are often fraught with analytic uncertainty. In three studies, we explore the effect of presenting probability assessments and analytic uncertainty through probability ranges. Participants were presented with mock intelligence forecasts that include narrative evidence as well as numerical probability assessments. Participants were sensitive to the ambiguity communicated through the confidence range. The narrative appeared to have a smaller effect on judgments when accompanied by a probability range as opposed to a point assessment. In one study, participants also thought that the probability range was more useful for decision making at a higher probability whereas the point estimate was more useful at a lower probability. When evaluating a forecast in hindsight, decisionmakers tended to report lower levels of blame and higher levels of source credibility for forecasts that reported ranges as compared to point assessments. These findings suggest that decisionmakers are not necessarily "ambiguity averse" in the forecasting context. Presenting ranges of probability may have distinct advantages as a way to communicate probability and analytic confidence to decisionmakers.

AB - How to assess and present analytic uncertainty to policymakers has emerged as an important topic in risk and policy analysis. Due to the complexity and deep uncertainty present in many forecasting domains, these reports are often fraught with analytic uncertainty. In three studies, we explore the effect of presenting probability assessments and analytic uncertainty through probability ranges. Participants were presented with mock intelligence forecasts that include narrative evidence as well as numerical probability assessments. Participants were sensitive to the ambiguity communicated through the confidence range. The narrative appeared to have a smaller effect on judgments when accompanied by a probability range as opposed to a point assessment. In one study, participants also thought that the probability range was more useful for decision making at a higher probability whereas the point estimate was more useful at a lower probability. When evaluating a forecast in hindsight, decisionmakers tended to report lower levels of blame and higher levels of source credibility for forecasts that reported ranges as compared to point assessments. These findings suggest that decisionmakers are not necessarily "ambiguity averse" in the forecasting context. Presenting ranges of probability may have distinct advantages as a way to communicate probability and analytic confidence to decisionmakers.

KW - Ambiguity

KW - Analytic uncertainty

KW - Forecasting

KW - Imprecise probability

KW - Risk communication

UR - http://www.scopus.com/inward/record.url?scp=77954093303&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77954093303&partnerID=8YFLogxK

U2 - 10.1111/j.1539-6924.2010.01384.x

DO - 10.1111/j.1539-6924.2010.01384.x

M3 - Article

VL - 30

SP - 987

EP - 1001

JO - Risk Analysis

JF - Risk Analysis

SN - 0272-4332

IS - 6

ER -