Variance and standard deviation

Two data sets can share a mean and have nothing else in common. The results $9, 10, 11$ and the results $2, 10, 18$ both have a mean of $10$, but the first is predictable and the second is all over the place. The measures of location from the previous lesson cannot see that difference — measures of spread exist for it.

The range

The simplest of them is the range: the distance between the largest and the smallest value.

$$R = x_{\max} - x_{\min}$$

It takes a second to compute and is worth about that much: it uses only two observations. A set where everything sits near the mean and one value runs off has the same range as a set spread evenly. A serious description needs a measure that looks at every observation.

Deviation from the mean

The natural idea: for each observation, work out how far it lies from the mean.

$$d_i = x_i - \bar{x}$$

Except that these deviations cannot simply be averaged. For every data set:

$$d_1 + d_2 + \dots + d_n = 0$$

The positive and negative deviations cancel exactly — which is really another way of saying what a mean is. Their average is always zero, so it is useless as a measure of spread.

The fix is the square: it removes the sign and, along the way, weights large deviations more heavily than small ones.

Variance

The variance is the mean of the squared deviations:

$$\sigma^2 = \frac{(x_1 - \bar{x})^2 + (x_2 - \bar{x})^2 + \dots + (x_n - \bar{x})^2}{n}$$

Three steps: find the mean, find the deviations, average their squares.

The same result comes out of the shortcut formula, which never writes a deviation down:

$$\sigma^2 = \frac{x_1^2 + x_2^2 + \dots + x_n^2}{n} - \bar{x}^2$$

Check it on the same data: $\tfrac{49 + 81 + 121 + 169}{4} - 10^2 = \tfrac{420}{4} - 100 = 105 - 100 = 5$. It agrees. This version is far quicker whenever the mean is not a whole number.

Standard deviation

The variance has one flaw: it lives in squared units. If the data is money, the variance comes out in "squared pounds", which compares to nothing. So we take the root:

$$\sigma = \sqrt{\sigma^2}$$

The standard deviation is in the same units as the data and says directly what order of magnitude the deviations from the mean are. Beware the popular shorthand "σ is the average distance from the mean" — it is not: the average distance is a different measure (the mean absolute deviation), while σ is the root of the mean of the squares, so it weighs large deviations more heavily and always comes out at least as big.

For the set in the example above, $\sigma = \sqrt{5} \approx 2.24$. That says the results stay within about two units of the mean of $10$ — roughly between $8$ and $12$.

The root almost never comes out "nice", and that is normal: a variance may be a whole number, a standard deviation usually is not. That is why $\sigma$ is quoted rounded, most often to two decimal places.

Same mean, different spread

Put two sets of five measurements side by side. The first:

$$8,\ 9,\ 10,\ 11,\ 12$$

The second has the same mean:

$$4,\ 7,\ 10,\ 13,\ 16$$

Both measures of spread:

$$\sigma_A^2 = \frac{4 + 1 + 0 + 1 + 4}{5} = 2, \qquad \sigma_A = \sqrt{2} \approx 1.41$$ $$\sigma_B^2 = \frac{36 + 9 + 0 + 9 + 36}{5} = 18, \qquad \sigma_B = \sqrt{18} \approx 4.24$$

The two means are identical and the standard deviation is three times larger in the second set. That is exactly what a description quoting only the mean throws away — and exactly why $\bar{x}$ and $\sigma$ are quoted together.

Population or sample

The variance formula sometimes carries the divisor $n-1$ instead of $n$:

$$s^2 = \frac{(x_1 - \bar{x})^2 + \dots + (x_n - \bar{x})^2}{n-1}$$

The difference is not cosmetic — it is about what the data is:

• population variance ($\sigma^2$, divisor $n$) — when the data is the entire group you are describing: every grade in the class, every measurement taken;
• sample variance ($s^2$, divisor $n-1$) — when the data is a sample drawn from something larger and the conclusion is meant to be about that larger thing.

Why $n-1$? The deviations are measured from the sample mean, and that always sits closer to the data than the unknown population mean does. The sum of squares therefore comes out slightly too small, and the smaller divisor corrects it. For large data sets the difference is negligible; for a handful of values it is not.

The symbols themselves say which version is meant: $\sigma^2$ and $\sigma$ are population measures, $s^2$ and $s$ sample ones. The exercises below use the population variance and deviation, dividing by $n$.

Shifting and rescaling the data

Two properties that save a lot of arithmetic:

• adding a constant to every observation changes nothing — not the range, not the variance, not the deviation, because the mean shifts along with the data and every $x_i - \bar{x}$ stays put;
• multiplying every observation by $k$ multiplies the standard deviation by $|k|$ and the variance by $k^2$.

Hence the practical rule: changing units (centimetres to metres) changes $\sigma$, but moving the point of reference does not.

Exercises

The three kinds of question match the three measures in this lesson. The Range prompt asks for the difference between the largest and the smallest value. The $\sigma^2$ prompt asks for the variance and the $\sigma$ prompt for the standard deviation — both for a population, that is with divisor $n$. A standard deviation is almost never rational, so type it to two decimal places, for instance 3.16; a more precise answer is accepted too. The same tolerance applies to the harder variance questions, where that value comes out fractional as well.

Practice

Work through a set of exercises — they get harder as you go. At the end you'll see your score and the mistakes worth reviewing.

Exercise 1 of 8Score: 0

Range: 17, 3, 7, 2, 17ResultCheck

Common mistakes

• Averaging the deviations without squaring — they sum to zero, so the answer would be 0 for every data set.
• Confusing the variance with the deviation — a question about $\sigma$ needs the root; $\sigma^2$ alone is not the answer yet.
• The wrong divisor — a population variance divides by $n$, not by $n-1$ and not by the number of distinct values.
• Rounding mid-computation — round only the final $\sigma$, otherwise the error accumulates.
• Measuring deviations from the first value or from the median — the reference point is always the mean.
• A negative variance — if you got one, a square went missing somewhere: a mean of squares cannot be negative.