[tahoe-dev] Bayesian Approach to Black Swans

[Rob Meijer]

On Sun, April 5, 2009 23:32, Josh Wilcox wrote:
>   I don't understand why Black Swans should be impossible to model
> effectively.
>   I've read neither "Fooled by Randomness" nor "The Black Swan", so maybe
> I
> don't know what a "Black Swan" is, but my naive impression is that they
> are
> rare-but-important-events.
>   If this is the case then mightn't we guesstimate their frequency with
> some, presumably low, confidence due to their elusive nature, and
> incorporate that belief into our model?
> --J
> P.S.--  I am assuming a Bayesian paradigm.


Risk analysis mostly uses a set of accepted probability density functions
that are used to approach and estimate the real probability density of a
particular impact based on a large number of measurements.
We know and accept the accepted probability density functions are
idealized distributions that only approach reality. We simply accept
the inaccuracy in the extremes of the function.
Many stochastic variables are composites of what not all components are
known. If no or very little measurements resulted from one or more of the
components, these components will add more to the inaccuracy of the
probability density used in calculations.

Lets take the example of a visit to the casino. If you take 10000 people,
send measure how much they own before going to the casino, and measure how
much they own after leaving the casino, you will get 10000 measurements
that you could use for modeling and infer a probability density from these
measurements, that you assume to be accurate. The measurements follow a
nice normal distributions so all seems quite OK and you are confident of
the result.

In doing so you will probably have missed however that some of the
casino's have a big jackpot slot machine that all didn't pay out, and half
of them all just happen to be just days away from hitting some major
psychological threshold in the amounth of money in the jackpot.

No after you have gotten the measurements you are asked to calculate the
probability someone walking into a casino has next month of hitting (what
you don't realize is) a slotmachine jackpot event.
You will look at the probability density derived from your measurements
and conclude a probability that will only be a pretty small fraction of
the real probability.

There are I hope you will realize many much less obvious components in
stochastic variables than the example above, making probability density
functions that are at the very heart of risk analysis pretty untrustworthy
in their extremes. Sure you can make a list of known real high impact
events upto things like Yellowstone erupting, bees going extinct or our
planet being hit by astroids and try to factor them in into your models.
Some may even be tempted to try and factor in probabilities for the
horseman of the apocalypse arriving, the invasion of earth by the Annunaki
people from Nibiru, or Surtr and Freyr getting into a big fight :-)

There will however still remain  hidden but important components to many
stochastic variables that you will be unable to reveal.

Rob