27 The Binomial and Normal Distributions
Bernoulli trials and the binomial distribution
A single patient’s ulcer either heals within the trial’s follow-up window or it does not. Coded \(1\) for healed and \(0\) otherwise, this is a Bernoulli trial: one outcome with a fixed probability \(p\) of success (the binary variable coding). Treat \(n\) patients this way, each an independent Bernoulli trial with the same success probability \(p\), and count the total number of successes. That count is a binomial random variable.
To find the probability of \(k\) successes, count the number of ways \(k\) successes can be arranged among the \(n\) trials, then multiply by the probability of one such arrangement. A single arrangement with \(k\) successes and \(n-k\) failures, in a particular order, has probability \(p^k(1-p)^{n-k}\) by the multiplication rule for independent events. The number of orders in which \(k\) successes can land among \(n\) trials is the binomial coefficient
\[ \binom{n}{k} = \frac{n!}{k!\,(n-k)!}, \]
read “\(n\) choose \(k\).”
Flipping a coin ten times
Flip a fair coin \(10\) times and count the heads. Each flip is a trial with \(p(H) = 0.5\), so the count \(X\) is \(\mathrm{Binomial}(10,\ 0.5)\) (see Figure 27.1). The probability of exactly \(5\) heads is \(\binom{10}{5}/2^{10} = 252/1024 \approx 0.246\). The probability of at most \(2\) heads is \((1 + 10 + 45)/1024 = 56/1024 \approx 0.055\).
from math import comb
n = 10
probs = [comb(n, k) / 2**n for k in range(n + 1)]
print(f"P(5 heads) = {probs[5]:.3f} P(X <= 2) = {sum(probs[:3]):.3f}")P(5 heads) = 0.246 P(X <= 2) = 0.055
scripts/03_statistics/06_binomial_and_normal/gen_binomial_and_normal.py.
Changing the coin changes the distribution (see Figure 27.2). The two are mirror images: a coin that lands heads \(80\%\) of the time gives heads counts that peak at \(8\), where the \(p(H) = 0.2\) coin peaks at \(2\).
scripts/03_statistics/06_binomial_and_normal/gen_binomial_and_normal.py.
The mean and variance of a binomial count
The mean and variance of a binomial random variable follow from the \(0\)/\(1\) mean and variance. Write \(X = Y_1 + \cdots + Y_n\), where \(Y_i\) is \(1\) if trial \(i\) succeeds and \(0\) otherwise. Each \(Y_i\) has mean \(p\) and variance \(p(1-p)\). Summing \(n\) of them adds the means. Independence lets the variances add as well.
\[ \mu_X = np, \qquad \sigma_X^2 = np(1-p), \qquad \sigma_X = \sqrt{np(1-p)}. \]
The device arm has a mean of \(39\) healed ulcers out of \(130\), with standard deviation about \(5.2\). The control arm has a mean of \(21\) out of \(115\), with standard deviation about \(4.1\). Dividing the standard deviation of the count by \(n\) gives the standard deviation of the healing rate: \(\sqrt{np(1-p)}/n = \sqrt{p(1-p)/n}\). This is the standard-error formula for a rate.
The Normal distribution
The normal distribution (also called the Gaussian distribution) is a symmetric, bell-shaped model for a continuous measurement. It is fixed by two parameters: the mean \(\mu\), which sets the center, and the standard deviation \(\sigma\), which sets the spread.
The model is specified by its density,
\[ f(x) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2}, \]
where \(x\) is a value of the measurement, \(e \approx 2.718\) is Euler’s number, \(\mu\) is the mean (the center of the distribution), and \(\sigma\) is the standard deviation (the spread). The exponent equals zero at \(x = \mu\) and becomes more negative as \(x\) moves away from \(\mu\) in either direction. The density is therefore highest at the mean and falls off symmetrically on both sides.
The height \(f(x)\) is not a probability. It shows how densely measurements concentrate near \(x\). The normal distribution is a probability density function (PDF): its height is never negative, and its total area is \(1\). A PDF gives probabilities by integration. The probability that \(X\) falls between \(a\) and \(b\) is the integral of the density over that range:
\[ P(a \le X \le b) = \int_a^b f(x)\,dx. \]
scripts/03_statistics/06_binomial_and_normal/gen_binomial_and_normal.py.
Cutting the curve at \(40\) and at \(60\) gives the two shaded areas in Figure 27.3. Integrating from the far left up to \(40\) gives a tail probability.
The cumulative distribution function
The cumulative distribution function (CDF) of \(X\) is \(F(x) = P(X \le x)\), the area under the density to the left of \(x\). For the normal model, \(F(x) = \int_{-\infty}^{x} f(t)\,dt\). The standard normal CDF is written \(\Phi\), so \(\Phi(z) = P(Z \le z)\).
scripts/03_statistics/06_binomial_and_normal/gen_binomial_and_normal.py.
Figure 27.4 plots \(\Phi\). It rises from \(0\) to \(1\). At \(z = 1.96\), \(\Phi(1.96) = 0.975\), so \(97.5\%\) of the area lies to the left.
Tail areas follow from \(\Phi\). The area to the right of \(z\) is \(1 - \Phi(z)\). The two-sided area beyond \(|z|\) is \(2\,(1 - \Phi(|z|))\), which is the p-value of a z-test (hypothesis test). The CDF also runs in reverse: \(\Phi(z) = 0.03\) gives \(z \approx -1.88\).
The mean and standard deviation define the curve (see Figure 27.5). The two curves have the same shape and width, but the second peak sits \(8\) units further right: changing \(\mu\) shifts the curve without changing its shape. On the right, \(N(50, 8)\) is compared with \(N(50, 4)\). Both peak at \(50\), but the second is narrower and taller, with peak height \(1/(4\sqrt{2\pi}) \approx 0.100\) against \(1/(8\sqrt{2\pi}) \approx 0.050\) for the first. Each curve encloses area \(1\), so a narrower curve must be taller. Changing \(\sigma\) stretches or compresses the curve around its center.
scripts/03_statistics/06_binomial_and_normal/gen_binomial_and_normal.py.
Move the sliders in Figure 27.6. The curve is the density \(f(x)\) of \(N(\mu, \sigma)\). The readout gives the peak height, which falls as \(\sigma\) grows, and two probabilities. The area within one standard deviation of the mean is about \(0.683\) for every choice of the sliders.
A normal distribution with mean \(\mu\) and standard deviation \(\sigma\) is written \(N(\mu, \sigma)\). The case \(\mu = 0\), \(\sigma = 1\) is the standard normal distribution, \(N(0, 1)\) (Diez et al. 2019).
Standard Normal Distribution
Every normal distribution has its own mean and standard deviation, so a table of normal areas would need a separate entry for each pair. The standard normal distribution, \(N(0, 1)\), removes that need. Any normal variable \(X\) can be converted to a standard normal variable \(Z\) by subtracting its mean and dividing by its standard deviation. Probabilities about \(X\) then become probabilities about \(Z\), and one table can be used for every case.
Measurements in different units and with different spreads become comparable once they are standardized.
The standard normal and the z-score
The z-score of a value \(x\) drawn from \(N(\mu, \sigma)\) is its distance from the mean in units of the standard deviation:
\[ z = \frac{x - \mu}{\sigma}, \]
where \(x - \mu\) is the signed distance from the mean. A z-score of \(1.5\) means the value lies \(1.5\) standard deviations above the mean, and a negative z-score means it lies below. An observation at the mean has \(z = 0\). Because the z-score removes both \(\mu\) and \(\sigma\), values from any normal distribution land on the same scale.
Take the model \(N(50, 8)\). The value \(58\) lies \(8\) units above the mean, one standard deviation, so
\[ z = \frac{58 - 50}{8} = 1. \]
The value \(40\) lies \(10\) units below the mean, so \(z = \frac{40 - 50}{8} = -1.25\). The z-score describes a value’s position. A probability needs the area of the standard normal curve beyond that position, which a table of standard normal areas or software such as scipy.stats.norm.cdf supplies.
The 68–95–99.7 rule
The area within \(1\), \(2\), and \(3\) standard deviations of the mean is the same for every normal distribution, because every normal curve is a rescaling of the standard normal.
def normal_pdf(x, m, s):
"""Normal probability density with mean m and standard deviation s."""
return np.exp(-0.5 * ((x - m) / s) ** 2) / (s * np.sqrt(2 * np.pi))
z = np.linspace(-4, 4, 4001)
pdf = normal_pdf(z, 0, 1)
print("area of the standard normal within k standard deviations of the mean:")
for k in [1, 2, 3]:
inside = np.abs(z) <= k
area = np.sum(pdf[inside]) * (z[1] - z[0])
print(f" within +/- {k} SD: {area*100:.1f}%")area of the standard normal within k standard deviations of the mean:
within +/- 1 SD: 68.3%
within +/- 2 SD: 95.4%
within +/- 3 SD: 99.7%
scripts/03_statistics/06_binomial_and_normal/gen_binomial_and_normal.py.
The areas are \(68.3\%\) within one standard deviation of the mean, \(95.4\%\) within two, and \(99.7\%\) within three (see Figure 27.7). Rounded, they give the 68–95–99.7 rule: about \(68\%\) of a normal distribution’s values fall within one standard deviation of the mean, about \(95\%\) within two, and about \(99.7\%\) within three. ::::: {.callout-important .knowledge-check icon=“false”} ## Knowledge check
A population of recovery times is modeled as normal with mean \(\mu = 50\) days and standard deviation \(\sigma = 8\) days. (a) What is the z-score of a recovery time of \(66\) days? (b) Using the \(68\)–\(95\)–\(99.7\) rule, about what fraction of patients recover within \(66\) days?
:::::
Normal approximation to the binomial
A binomial probability with a large \(n\) is a long sum. The probability of \(45\) or fewer heads in \(100\) fair flips adds \(46\) terms. The normal curve gives an answer from one table lookup when the rule of thumb below is met. For a binomial count, the normal model uses the binomial’s own mean and standard deviation:
\[ \mu = np, \qquad \sigma = \sqrt{np(1-p)}. \]
At \(n = 10\) the bars pile up against zero, and the normal curve fits them poorly. As \(n\) grows, the bars fill out a symmetric bell, and the normal curve fits them closely (see Figure 27.8).
scripts/03_statistics/06_binomial_and_normal/gen_binomial_and_normal.py.
NoteNormal approximation of the binomial distribution
As a general rule, the binomial is well approximated by a normal curve when \(np \geq 10\) and \(n(1-p) \geq 10\). The normal curve takes the binomial’s mean and standard deviation.
Take \(100\) flips of a fair coin. Here \(np = 50\) and \(n(1-p) = 50\), so the rule is met, and the normal model is \(N(50,\ 5)\) (see Figure 27.9).
n, p = 100, 0.5
mean = n * p
sd = (n * p * (1 - p)) ** 0.5
print(f"np = {mean:.1f}, n(1-p) = {n*(1-p):.1f} (both >= 10)")np = 50.0, n(1-p) = 50.0 (both >= 10)
scripts/03_statistics/06_binomial_and_normal/gen_binomial_and_normal.py.
Find \(P(X \le 45)\). The exact binomial sum is \(0.184\). The count takes whole values, so the normal cutoff is moved halfway to the next value, to \(45.5\). The normal curve then gives \(z = (45.5 - 50)/5 = -0.9\) and a probability of \(0.184\).
Application: the Dermagraft trial
Take the healing rate of the device arm as known, \(p = 0.300\), with \(130\) patients. Then \(np = 39.0\) and \(n(1-p) = 91.0\), so the rule is met, and the normal model is \(N(39.0,\ 5.22)\).
The probability that \(39\) or fewer patients heal is \(0.543\) exactly. The normal curve, with the cutoff moved to \(39.5\), gives \(z = (39.5 - 39.0)/5.22 \approx 0.096\) and a probability of \(0.538\), within \(0.005\) of the exact value.
References
Diez, David M., Mine Çetinkaya-Rundel, and Christopher D. Barr. 2019. OpenIntro Statistics. 4th ed. OpenIntro. https://www.openintro.org/book/os/.