Normal approximation to the Beta posterior distribution

We have seen that the Beta distribution (s+1, n-s+1) provides an estimate of the binomial probability p when we have observed s successes in n independent trials, and assuming a prior Uniform(0,1) distribution. The posterior density has the function:

Taking logs gives:

and

We first find our best estimate q0 of q:

which gives the intuitively encouraging answer:

i.e. our best guess for the binomial probability is the proportion of trials that were successes. Next we find the standard deviation s for the Normal approximation to this Beta distribution:

which gives:

(1)

and so we get the approximation:

(2)

Equation 1 for s allows us some useful insight into the behaviour of the Beta distribution. We can see in the numerator that the spread of the Beta distribution, and therefore our measure of uncertainty about q's true value, is a function of our best estimate for q. The function [q0 (1-q0)] is at its maximum when q0 = ½ , so for a given set of trials n, we will be more uncertain about the true value of q if it is close to ½ than if it was closer to 0 or 1. Looking at the denominator we see that the degree of uncertainty, represented by s, is proportional to n-½ . We will see time and again that the level of uncertainty of some parameter is inversely proportional to the square root of the amount of data available. Note also that Equation 2 is exactly the same as the classical statistics result of making the Normal approximation to the Binomial.

But when is this quadratic approximation to L(q), i.e. the Normal approximation to f(q), a reasonably good fit? The mean m and variance V of a Beta(s+1,n-s+1) distribution are as follows:

Comparing these identities with Equation 2, we can see that the Normal approximation works when s and (n-s) are both sufficiently large that adding 1 to s and adding 3 to n proportionally have little effect, i.e. when:

and

Normal approximation to the Beta posterior distribution

Navigation