Introduction to distributional approximation
Distributional approximation replaces a complicated probability law by a simpler one. Sums of Bernoulli variables provide examples in which binomial, Poisson and normal approximations can be compared directly.
Sums of Bernoulli variablesContents
Let be independent Bernoulli variables with success probabilities . Their sum has the Poisson–binomial distribution:
When all equal , . Two further extensions are a random number of trials, , and dependent indicators, such as overlapping runs of consecutive successes.
Comparing approximationsContents
The three approximations below match the theoretical conditional mean: , , and a normal law with variance . For the normal approximation, mass at each integer is defined by a continuity correction:
The probabilities are drawn once using a fixed seed. The exact distribution is computed by repeated convolution.
| Approximation | Total variation | Kolmogorov distance |
|---|
Distances include mass outside ; the plot shows the central range. These curves use theoretical moments, whereas the simulations below fit parameters from the observations.
- Rare events: , with . The mean remains near 3; the Poisson approximation improves as grows. A bounded variance does not give a normal limit.
- Near one half: , with . Normal and binomial approximations work well. The Poisson variance is too large.
- Strong heterogeneity: . Typical variances are for the exact law, for the mean-matched binomial, and for Poisson.
The binomial variance excess is exactly the heterogeneity of the success probabilities.
Refinements
A skewness correction uses , , and in place of . A saddlepoint approximation uses the cumulant generating function :
For at , the exact mass is 0.02216; the discrete normal gives 0.01926, the skewness correction 0.02354, and the saddlepoint approximation 0.02246. The two refinements approximate individual masses and need not form normalised probability distributions.
Monte Carlo goodness of fitContents
The original simulations use 100 observations per repetition, 100 repetitions at each , and 9,999 parametric bootstrap samples per test, with parameter re-estimation. A fresh probability vector is drawn for each repetition and held fixed across its observations. Shaded bands are approximate pointwise 95% confidence intervals for the mean -values.
These averages summarise repeated tests. They are not -values for a combined test, and changing the controls above does not change these saved figures.
Quantitative guaranteesContents
For independent Bernoulli sums, Le Cam’s inequality gives
For with fixed , the right-hand side is . The Chen–Stein bound improves this by the factor .
The Berry–Esseen inequality gives, when ,
This compares with a continuous normal in Kolmogorov distance. Total variation between a discrete law and a continuous law is 1.
Random sums and dependence
For an independent random number of ..d. trials,
Replacing by misses the second term. If , then ; if , then .
For overlapping blocks of consecutive successes in ..d. trials, the mean is and
The additional covariance terms require approximation methods that account for dependence.
Probability metricsContents
An integral probability metric compares expectations over a class of integrable test functions:
| Distance | Test functions | Property |
|---|---|---|
| Total variation | Indicators of measurable sets | Largest difference in event probabilities |
| Kolmogorov | Indicators of half-lines | Largest CDF gap; |
| Wasserstein–1 | 1-Lipschitz functions | Transport cost; finite first moments |
| Bounded–Lipschitz | Metrizes weak convergence on | |
| Zolotarev | derivatives, with -Hölder mth derivative; | Under finite absolute sth moments, finiteness requires matching moments through order |
For integer-valued laws, . On , . A general test class gives an extended pseudo-metric and may fail to distinguish different laws.
Weak convergence and counterexamples
Weak convergence means convergence of expectations for every bounded continuous function. It does not imply convergence in every metric.
- For , but .
- The uniform law on approaches with , while .
- For versus , but .
If one law has a density bounded by and both have finite first moments, a useful comparison is .