Bayesian Probability Theory

Frequency Interpretation: Counting

Probabilities –> relative frequencies
The probability of the event E: $\lim_{n \to \infty} k_n (E) / n$

  • Problem:
    1. Frequency Interpretation is Circular (unlikely occurance)
    2. Reference Class Problem (No identical similar)
    3. Limited to I.I.D. (non-i.i.d. data)

Objective Interpretation: Uncertain Events

Objectivist probabilities –> real aspects of the world

Axioms 3.1 (Kolmogorov ‘s axioms of probability theory)
Let $\Omega$ be the sample space. Events are subsets of $\Omega$.
If A and B are events, then A $\cap$ B, A $\cup$ B, and A \ B are events.
$\Omega$ and {} are events.
There is a function p which assigns nonnegative reals, called proabilities, to each event.
p($\Omega$) = 1, p({}) = 0.
p(A $\cup$ B) = p(A) + p(B) - p(A $\cap$ B).
For a decreasing sequence A1 $\supset$ A2 $\supset$ A3 … of events with $\bigcap_n$ An = {} we have $\lim_{n \to \infty}$ p(An) = 0.

Definition 3.2 (Conditional probability)
If A and B are events with p(A) > 0, then $p (B | A) := \frac{p (A \cap) B}{p(A)}$

p($\cdot$ | A) is also a probability measure, if p($\cdot$) satisfies the Kolmogorov axioms.

Theorem 3.3 (Bayes’ rule 1)
If A and B are events with p(A) > 0 and p(B) > 0, then $p(B | A) = \frac{p(A | B) p(B)}{p(A)}$

Subjective Interpretation: Degree of Belief

Axioms 3.4 (Cox’s (1946) axioms for beliefs)
The degree of belief in event B given that event A occurred: function Bel(B|A).