Planning the commute
Ten timed runs to the office: 25, 26, 26, 27, 28, 28, 29, 30, 31 and 90 minutes, the last one thanks to a crash on the ring road. The mean is 340 : 10 = 34 minutes, the median (28 + 28) : 2 = 28. The median says what an ordinary morning costs; the mean smears one accident thinly across nine normal days. You leave by the median and keep a separate margin for the day that goes wrong, instead of budgeting six extra minutes every morning for something that happened once.
Mean, median and mode
Choosing a courier
Two firms deliver a parcel in 3 days on average. The first: 2, 3, 3, 3, 4 days — variance (1 + 0 + 0 + 0 + 1) : 5 = 0.4, deviation 0.63 of a day. The second: 1, 1, 3, 5, 5 days — variance 16 : 5 = 3.2, deviation 1.79. A present that has to arrive on a given day goes with the first, even though the price list and the advert quote the very same average for both.
Variance and deviation
A chart in the newspaper
A story about a poll shows two bars, 42 and 46 per cent, on a vertical axis starting at 40. The drawn heights are 2 and 6 units, so the leader’s bar is three times taller. The true ratio is 46 : 42, a lead of 9.5 per cent of the value — the drawing overstates it more than thirtyfold. One glance at the number at the bottom of the axis settles whether the picture can be trusted.
Presenting data
How strong a password is
A four-digit PIN is 10⁴ = 10,000 possibilities — a phone locks after three attempts, but a stolen database of such codes falls in a fraction of a second. Eight lower-case letters give 26⁸ ≈ 209 billion combinations, and eight characters drawn from 62 (lower case, upper case, digits) give 62⁸ ≈ 218 trillion, over a thousand times as many. Every character you add multiplies that count by the size of the alphabet rather than adding to it.
Combinatorics See the lesson: Powers
A lottery ticket
Six numbers out of 49 make C(49, 6) = 13,983,816 equally possible tickets, so P(jackpot) = 1 : 13,983,816 ≈ 0.0000072%. One ticket a week for 50 years is 2,600 draws, and the chance of ever hitting it — taken through the opposite event, 1 minus the probability of 2,600 misses — comes to about 0.019%, fewer than two chances in ten thousand. That number is nowhere on the slip.
Classical probability
Reading a screening result
A test detects the disease in 99 of every 100 patients who have it and misfires on 2 of every 100 who do not, while 1 person in 100 is actually ill. A positive result reaches 0.01 · 0.99 = 0.0099 of the ill and 0.99 · 0.02 = 0.0198 of the healthy, so 2.97 per cent of everyone tested. The probability of being ill after a positive result is 0.0099 / 0.0297 — exactly one third, not 99 per cent. That is why a positive screening result is always repeated with a confirmatory test.
Conditional probability
Inspecting a batch of goods
From a batch in which 5 per cent of units are faulty, a storekeeper pulls 10 for inspection. The probability that none of them turns out faulty is 0.95 to the tenth power, that is 0.599 — so almost 40 per cent of such inspections catch at least one defect. Had 20 per cent been faulty, a sample of ten would pass the batch only 11 per cent of the time, and that is the whole mathematics behind choosing a sample size.
Trees and the Bernoulli scheme
A lottery ticket
A ticket costs 5. The top prize of 1000 comes up with probability 1/2000 and a consolation 50 with probability 1/200. The expected payout is 1000/2000 + 50/200 = 0.50 + 0.25 = 0.75, so the player’s expected result is 0.75 − 5 = −4.25 per ticket. A hundred tickets a year means an expected loss of 425, and no run of luck changes that — it only changes when it shows.
Random variables
Overbooking a flight
An airline sells 210 tickets for 200 seats because a passenger shows up with probability 0.95. The number who show up is binomial with E(X) = 210 · 0.95 = 199.5 and variance 210 · 0.95 · 0.05 = 9.975, that is a standard deviation of 3.16 passengers. On average half a seat stays free, but a spread of three passengers means overbooking happens on a sizeable share of flights — and that is the number the airline is knowingly buying.
Binomial and Poisson
Manufacturing tolerance
A filling machine puts an average of 500 g into a packet with a standard deviation of 2 g. The three-sigma rule says 99.7 per cent of packets fall between 494 and 506 g. If the standard allows a 5 g deviation, that is 2.5 sigmas, then 1.24 per cent of production falls outside tolerance — at 20,000 packets a day that is 248 units to reject, and that is the number a decision to recalibrate rests on.
The normal distribution
A poll with a margin of error
A pollster surveys 1067 people and reports 34 per cent support "with a margin of error of 3 percentage points". That number is not a convention: at the worst possible spread of answers the standard error is √(0.25/1067) ≈ 0.0153, and the margin at a 95 per cent confidence level is 1.96 · 0.0153 ≈ 0.030, i.e. 3.0 points. The confidence interval therefore runs from 31 to 37 per cent — and it is the interval, not the single number, that is the result of the poll.
Estimation and confidence intervals
An A/B test
Variant A converts at 4.0 per cent. Variant B gets 2500 views and converts 120 times, that is 4.8 per cent. Under the hypothesis that nothing changed, the standard error is √(0.04 · 0.96 / 2500) ≈ 0.0039, so z = (0.048 − 0.040)/0.0039 ≈ 2.04. That clears 1.96, so at α = 0.05 the null hypothesis is rejected — but only just, and the rollout decision should say so.
Hypothesis testing
Flat price against floor area
Sixty transactions in one district give the line ŷ = 9.2x + 45, where x is the floor area in square metres and ŷ the price in thousands. The model values a 50-square-metre flat at 9.2 · 50 + 45 = 505 thousand. The coefficient of determination is 0.78, so floor area explains 78 per cent of the variation in price — the other 22 per cent is the floor, the condition, the aspect, and how much of a hurry the seller was in.
Correlation and regression

Branch formulas

Branch: Statistics and probability

Mean, median and mode

Arithmetic mean
$\bar{x} = \frac{x_1 + x_2 + \dots + x_n}{n}$
the sum of all the data divided by how many there are
Total from the mean
$x_1 + x_2 + \dots + x_n = n \cdot \bar{x}$
this is how a missing value is recovered when the mean is known
Median — an odd sample
$\mathrm{Me} = x_{\left(\frac{n+1}{2}\right)} \quad (n \ \text{odd})$
once sorted, it is exactly the middle number
Median — an even sample
$\mathrm{Me} = \frac{x_{\left(\frac{n}{2}\right)} + x_{\left(\frac{n}{2}+1\right)}}{2} \quad (n \ \text{even})$
the mean of the two middle values
Mode
$\mathrm{Mo} = \text{the value with the highest frequency}$
the number that occurs most often — there may be none at all
Weighted mean
$\bar{x}_w = \frac{w_1 x_1 + w_2 x_2 + \dots + w_k x_k}{w_1 + w_2 + \dots + w_k}$
every value counts as many times as its weight says

Variance and deviation

Range
$R = x_{\max} - x_{\min}$
the simplest measure of spread — it looks only at the two extremes
Deviation from the mean
$d_i = x_i - \bar{x}, \qquad d_1 + d_2 + \dots + d_n = 0$
the deviations always sum to zero, which is why they cannot be averaged as they are
Variance
$\sigma^2 = \frac{(x_1 - \bar{x})^2 + (x_2 - \bar{x})^2 + \dots + (x_n - \bar{x})^2}{n}$
the mean of the squared deviations from the mean
Standard deviation
$\sigma = \sqrt{\sigma^2}$
the square root of the variance — back in the units of the data
Shortcut formula
$\sigma^2 = \frac{x_1^2 + x_2^2 + \dots + x_n^2}{n} - \bar{x}^2$
the mean of the squares minus the square of the mean — quicker to compute
Sample variance
$s^2 = \frac{(x_1 - \bar{x})^2 + \dots + (x_n - \bar{x})^2}{n-1}$
divisor n − 1, when the data is a sample drawn from a larger population

Presenting data

Total of the frequencies
$\sum_{i} n_i = n$
the table check: class counts must add up to the sample size
Relative frequency
$w_i = \frac{n_i}{n}$
the class's share; times 100 it is a percentage
Angle of a pie slice
$\alpha_i = 360^\circ \cdot \frac{n_i}{n}$
a full circle is 360°, so the angle is the share in degrees
Histogram class width
$h = \frac{x_{\max} - x_{\min}}{k}$
k — the number of classes; the range of the variance lesson cut into equal pieces
Frequency density
$g_i = \frac{n_i}{h_i}$
the bar height for UNEQUAL classes — not the count itself
Apparent ratio on a truncated axis
$\frac{H_B}{H_A} = \frac{x_B - c}{x_A - c}$
c — where the axis starts; at c = 0 it collapses to x_B / x_A

Combinatorics

Rule of product
$N = k_1 \cdot k_2 \cdot \ldots \cdot k_m$
a choice made in several independent steps — multiply the counts
Factorial
$n! = 1 \cdot 2 \cdot 3 \cdot \ldots \cdot n, \qquad 0! = 1$
the product of the first n naturals; 0! equals 1 by definition
Permutations
$P_n = n!$
the number of orderings of all n items
Arrangements without repetition
$\frac{n!}{(n-k)!} = n(n-1)\ldots(n-k+1)$
choose k out of n, order matters, nothing repeats
Arrangements with repetition
$n^k$
k times over, we choose from the same n options
Combinations
$\binom{n}{k} = \frac{n!}{k! \, (n-k)!}$
choose k out of n, order does NOT matter
Symmetry of combinations
$\binom{n}{k} = \binom{n}{n-k}$
choosing k items is the same as discarding the other n − k
Pascal's rule
$\binom{n}{k} = \binom{n-1}{k-1} + \binom{n-1}{k}$
every entry of the triangle is the sum of the two above it
The binomial theorem
$(a+b)^n = \sum_{k=0}^{n} \binom{n}{k} a^{n-k} b^k$
the coefficients of the expansion are row n of Pascal's triangle

Classical probability

Classical probability
$P(A) = \frac{|A|}{|\Omega|}$
favourable outcomes over the number of all possible ones
Range of a probability
$0 \leq P(A) \leq 1$
never below 0 and never above 1
Impossible and certain events
$P(\varnothing) = 0, \qquad P(\Omega) = 1$
edge cases of the definition, not results of a computation
Complementary event
$P(A') = 1 - P(A)$
counting "not A" is often easier than counting A
As a percentage
$P(A) \cdot 100\%$
the same probability expressed in percent
Independent events
$P(A \cap B) = P(A) \cdot P(B)$
multiply the probabilities — only when one does not affect the other

Conditional probability

Conditional probability
$P(A|B) = \frac{P(A \cap B)}{P(B)}$
defined only for P(B) > 0
Multiplication rule
$P(A \cap B) = P(B) \cdot P(A|B)$
the definition times P(B)
Counting form
$P(A|B) = \frac{|A \cap B|}{|B|}$
when the outcomes are equally likely
Law of total probability
$P(B) = \sum_{i=1}^{n} P(A_i) \cdot P(B|A_i)$
A₁, …, Aₙ split Ω into disjoint cases
Bayes’ theorem
$P(A_k|B) = \frac{P(A_k) \cdot P(B|A_k)}{\sum_{i=1}^{n} P(A_i) \cdot P(B|A_i)}$
reverses the condition: P(B|A) into P(A|B)
Independence
$P(A|B) = P(A) \iff P(A \cap B) = P(A) \cdot P(B)$
the condition changes nothing about A

Trees and the Bernoulli scheme

Multiplying along a branch
$P(\text{path}) = p_1 \cdot p_2 \cdot \ldots \cdot p_n$
the probability of one path of the tree
Adding across paths
$P(A) = \sum_{\text{paths} \subset A} P(\text{path})$
an event is a set of paths, so they add
Union of events
$P(A \cup B) = P(A) + P(B) - P(A \cap B)$
the overlap is counted twice, so subtracted once
Disjoint events
$A \cap B = \varnothing \implies P(A \cup B) = P(A) + P(B)$
nothing to subtract
Bernoulli scheme
$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$
k successes in n independent trials
At least one success
$P(X \geq 1) = 1 - (1-p)^n$
through the complement, with no summing

Random variables

Random variable
$X: \Omega \to \mathbb{R}$
a function attaching a number to an outcome
Distribution
$\sum_{i} p_i = 1, \qquad p_i = P(X = x_i)$
a table of values and their probabilities
Expected value
$E(X) = \sum_{i} x_i \, p_i$
a mean weighted by probabilities
Variance of a variable
$\sigma^2(X) = \sum_{i} \big(x_i - E(X)\big)^2 p_i$
mean squared deviation from E(X)
Computational form
$\sigma^2(X) = E(X^2) - \big(E(X)\big)^2$
one pass over the table instead of two
Standard deviation
$\sigma(X) = \sqrt{\sigma^2(X)}$
in the same unit as X
Distribution function
$F(x) = P(X \leq x)$
non-decreasing, from 0 to 1
Linear transformation
$E(aX + b) = a\,E(X) + b, \qquad \sigma^2(aX + b) = a^2 \sigma^2(X)$
a shift does not change the spread

Binomial and Poisson

Binomial distribution
$X \sim B(n, p), \qquad P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$
the formula derived in the Bernoulli lesson
Expected value
$E(X) = np$
trials times the chance of success
Variance
$\sigma^2(X) = np(1-p)$
largest at p = 1/2
At least one success
$P(X \geq 1) = 1 - (1-p)^n$
through the complement
Poisson distribution
$X \sim \text{Poiss}(\lambda), \qquad P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}$
rare events in a fixed window
Poisson parameters
$E(X) = \sigma^2(X) = \lambda$
one number describes the whole distribution
The limiting passage
$\lambda = np$
n large, p small, the product fixed

The normal distribution

The normal density
$f(x) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-\frac{(x-\mu)^2}{2\sigma^2}}$
the area under the whole curve is 1
Notation
$X \sim N(\mu, \sigma)$
μ shifts the curve, σ widens it
Standardisation
$z = \frac{x - \mu}{\sigma}$
how many deviations x lies from the mean
The standard distribution
$Z \sim N(0, 1)$
one distribution instead of an infinite family
The one-sigma rule
$P(|X - \mu| < \sigma) \approx 0.68$
two thirds of the population
The three-sigma rule
$P(|X - \mu| < 2\sigma) \approx 0.95, \qquad P(|X - \mu| < 3\sigma) \approx 0.997$
beyond three sigmas lie 3 cases in 1000
Central limit theorem
$\frac{\bar{X} - \mu}{\sigma / \sqrt{n}} \xrightarrow[n \to \infty]{} N(0, 1)$
a sample mean tends to a normal distribution
Normal approximation
$B(n, p) \approx N\big(np,\ \sqrt{np(1-p)}\big)$
usable once np(1 − p) ≥ 10

Estimation and confidence intervals

Point estimator of the mean
$\hat{\mu} = \bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i$
one number computed from the sample
An unbiased estimator
$E(\hat{\theta}) = \theta$
it does not err systematically — it errs at random
Sample variance
$s^2 = \frac{1}{n-1}\sum_{i=1}^{n} (x_i - \bar{x})^2$
the n − 1 divisor is the correction for unbiasedness
Standard error of the mean
$\mathrm{SE}(\bar{x}) = \frac{\sigma}{\sqrt{n}}$
the spread of the MEAN, not of the data
Confidence interval for a mean
$\bar{x} \pm z \cdot \frac{\sigma}{\sqrt{n}}$
z = 1.64 · 1.96 · 2.58 for 90 · 95 · 99 per cent
Margin of error
$d = z \cdot \frac{\sigma}{\sqrt{n}}$
half the width of the interval
Required sample size
$n = \left(\frac{z\sigma}{d}\right)^{2}$
the same formula read from the d end

Hypothesis testing

Null and alternative hypothesis
$H_0: \mu = \mu_0 \qquad H_1: \mu \neq \mu_0$
H₀ always carries the equality — that is the computable one
Test statistic for a mean
$z = \frac{\bar{x} - \mu_0}{\sigma / \sqrt{n}}$
how many standard errors the sample sits from the hypothesis
Decision rule (two-sided test)
$|z| > z_{\alpha/2} \;\Longrightarrow\; \text{reject } H_0$
z_{α/2} = 1.96 for α = 0.05
The p-value
$p = P\big(|Z| \geq |z| \;\big|\; H_0\big)$
the probability of SUCH DATA given H₀ — not of H₀ given the data
Type I error
$\alpha = P(\text{reject } H_0 \mid H_0 \text{ true})$
a false alarm; fixed in advance
Type II error
$\beta = P(\text{fail to reject } H_0 \mid H_1 \text{ true})$
missing an effect that is really there
Power of a test
$1 - \beta$
the chance of detecting an effect that exists

Correlation and regression

Sum of products of deviations
$S_{xy} = \sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})$
positive when the deviations lean the same way
Pearson correlation coefficient
$r = \frac{S_{xy}}{\sqrt{S_{xx} \cdot S_{yy}}}$
always between −1 and 1; no units
Least-squares line
$a = \frac{S_{xy}}{S_{xx}}, \qquad b = \bar{y} - a\bar{x}$
the line always passes through the point (x̄, ȳ)
Fitted value
$\hat{y} = ax + b$
the model’s prediction at a given x
Residuals
$e_i = y_i - \hat{y}_i, \qquad \sum_{i=1}^{n} e_i = 0$
it is their squares the method minimises
Coefficient of determination
$r^{2}$
the fraction of the variation in y the model explains
Significance test for a correlation
$t = \frac{r\sqrt{n-2}}{\sqrt{1-r^{2}}}$
t distribution with n − 2 degrees of freedom

Luminous histogram bars settling into the soft bell of a distribution, one outlying point brighter than the rest, against a cosmic nebula in violet and magenta

Statistics and probability

The average person does not exist

All of descriptive statistics rests on a single trade: you give up information about a crowd and receive one number in return. The trade is often worth making, but it is never free — and it pays to know exactly what gets lost in it. From the mortality tables of 1662, through Quetelet's average man, to the aircraft nobody ever saw.

16 min