Modeling and Coping with Extremely Rare or Adverse Events

Abstract

Extremely rare events occur in several areas of science and engineering. Some examples are huge movements in the prices of assets and commodities; and weather phenomena such as high winds and hurricanes. Attempts to model these kinds of events using the law of large numbers often fail because "rare" events seem to occur *far more frequently* than a Gaussian distribution would suggest. In some areas, such as clinical trials of drug candidates, a "law of large numbers" will not apply because the available data is far too small to permit the drawing of confident conclusions. For instance, one may have to estimate a few dozen parameters on the basis of a few dozen samples (whereas prudence would suggest at least a few thousand samples -- but these are simply not available). 

In this talk we give a *very elementary introduction* to some theoretical methods for modeling and coping with extremely rare or adverse events. To handle rare events, we suggest the use of "heavy tailed" random variables, which have finite first moment but infinite variance. The form of the "law of large numbers" for such variables, and how these affect the design process, will be discussed. To model adverse events with limited data, we suggest the use of "worst case probability distributions" which can be formulated as a linear programming problem, and often leads to surprisingly good insights.