Knowledge Content Library

Estimating Predictive Probability of Success

By Shaun Comfort, Roche-Genetech

Abstract: This article illustrates how Kahneman-Tversky’s (KT) original reference class forecasting (RCF) for calibrating subjective forecasts can be reformulated using the language of Bayesian inference. Shaun Comfort shows a simple implementation for estimating the probability of success for Bernoulli outcomes such as clinical trials, contract bids, and medical devices. The approach uses the Beta conjugate model. The reference class distribution becomes an informative "prior” and the team forecast is treated as new data. The predictive validity determines the effective sample size weight for the new data, in order to generate a posterior probability of success distribution. The resulting posterior mean is identical to the RCF corrective procedure point estimate. In addition, the Bayesian implementation provides the entire posterior probability distribution, from which useful statistics such as credible intervals can be calculated with ease. This approach can be a useful method for individuals external to development teams responsible for capturing, tracking, calibrating, and presenting forecasts for decision makers, such as portfolio leaders or decisional analysis professionals.

Go to article.

Keywords: analysis and modeling anamod, risk and uncertainty riskunc, expert interviews expint, probability assessment probass, value of information voi, cognitive bias cogbias, bayes

   Email | Print