The normal distribution and the three-sigma rule

The previous lesson ended on the continuous variable and a note about densities. There is one density more important than all the others put together — and the reason it is important comes halfway through this lesson.

The Gaussian curve

A random variable has a normal distribution with parameters $\mu$ and $\sigma$ when its density is

We write this as $X \sim N(\mu, \sigma)$. The formula need not be memorised — what matters is understanding what its two numbers do.

• $\mu$ slides the curve. It is simultaneously the mean, the median and the mode, because the bell is symmetric.
• $\sigma$ widens the curve. It is the distance from the peak to the inflection point — where the slope stops steepening and starts flattening.

The area under the whole curve is always $1$, so a wider curve has to be lower. The probability of an interval is the area under the curve above it, that is a definite integral:

That integral has no elementary form — and that is the only reason a normal distribution is read off tables, or off the rule below, instead of being computed.

Standardisation

There are infinitely many normal distributions, one for every pair $(\mu, \sigma)$. They need not all be tabulated, because the substitution

reduces every one of them to a single case. The variable $Z$ then has the standard distribution $N(0, 1)$: mean zero, deviation one. The number $z$ itself carries a very useful meaning: it says how many standard deviations the observation lies from the mean.

The three-sigma rule

Three numbers are worth knowing by heart, because they replace tables in most situations:

The rule also works read backwards, and that is how practice most often asks about it: since $95$ per cent of results lie in $\mu \pm 2\sigma$, the interval holding $95$ per cent of packets reaches $\pm 4$ grams from the mean.

Where the curve comes from

Why do so many different quantities share exactly this shape? Two limit theorems answer that.

The law of large numbers says where a sample mean is heading: as the number of observations grows, it approaches the expected value. It is what justifies the frequency of the classical-probability lesson approximating a probability at all.

The central limit theorem says how it gets there:

The sum or the mean of many independent quantities with finite variance is approximately normal — whatever distribution each of them has on its own. That is the real answer to “why is everything normal”: a person’s height, a measurement error or a commute time is the resultant of hundreds of small independent causes, and a resultant of such terms is normal almost always.

The denominator $\sigma/\sqrt{n}$ is worth noticing: the spread of the mean shrinks like the square root of the number of observations. To halve the error you must gather four times as much data.

When the bell curve does not hold

The theorem has hypotheses and they do get broken. Three situations are worth recognising:

• Fat-tailed distributions. Wealth, city sizes, daily stock returns, article view counts — there a single observation can exceed the mean a hundredfold. Under a normal distribution that would be practically impossible: a $6\sigma$ deviation has a probability of about one in a billion, and on the stock market such days come along every few years.
• Dependent terms. The theorem requires independence. When observations influence one another — a panic sell-off, an epidemic — their sums have no reason to be normal.
• One dominant term. If the resultant is really one cause plus noise, the shape comes from that cause and not from a limit theorem.

The practical conclusion is that normality has to be argued for, not assumed by default. A normal model applied to a fat-tailed distribution systematically understates the risk of extreme events — and that is the costliest error in this whole branch.

Approximating the binomial

The central limit theorem also acts on the distribution of the previous lesson. The number of successes is a sum of $n$ independent trials, so for large $n$

The practical criterion is $np(1-p) \geq 10$. For $n = 50$ and $p = \tfrac{1}{2}$ that comes to $12.5$, so the approximation is allowed — and the drawing shows it. When $p$ is very small, however, the product stays small even for enormous $n$, and there the right approximation is the Poisson distribution rather than the normal one.

Exercises

The set drills three skills. The $z$ prompt gives $x$, $\mu$ and $\sigma$ and asks for the standardisation, that is for the number of standard deviations between the observation and the mean. The prompt with an interval ending in an arrow and a per cent sign gives $\mu$, $\sigma$ and the two ends and asks what percentage of the population lies there — you read it off the 68–95–99.7 rule by adding bands.

The third prompt reads the same rule backwards: it gives the percentage and $\sigma$ and asks for $a$, the reach of the interval $\mu \pm a$. That is the version practice asks about — “what range holds 95 per cent of the results” — and that is why it stands here on its own.

In all three cases the answer is a number, not a fraction.

The generator does not ask you to judge whether a normal model may be assumed — that is a verdict, not a number. The three situations where the answer is “no” are listed in the section on fat tails.

Common mistakes

• Assuming normality without argument — fat-tailed distributions look perfectly ordinary in a small sample, and a normal model then understates the risk of extremes by orders of magnitude.
• Reading $\sigma$ as the width of the whole distribution — sigma is the distance to the inflection point; the real reach of the data is closer to $6\sigma$.
• Dividing by the variance instead of the deviation — standardisation has $\sigma$ in the denominator, not $\sigma^2$; the slip changes the answer by a factor of $\sigma$.
• Applying the three-sigma rule to an asymmetric interval without splitting it into bands — the interval from $-\sigma$ to $2\sigma$ is $34\% + 34\% + 13.5\%$, that is $81.5\%$, not the average of 68 and 95.
• Approximating a binomial by a normal at small $np(1-p)$ — for rare events the right model is the Poisson distribution; the normal curve there even produces negative estimates.
• Confusing the law of large numbers with a “law of averages” — the mean approaches $\mu$ because later results dilute the deviations, not because nature evens them out. The coin still does not remember previous tosses.