What is a random variable?

It is a function attaching a number to every outcome of an experiment. Rolling two dice has pairs of faces for outcomes, but what usually interests us is their sum — and that sum is the random variable. The name misleads: a random variable is neither a variable in the algebraic sense nor a random number, but a recipe for translating an event into a number.

What is the distribution of a random variable?

A list of every value the variable can take together with the probability of taking it. For a discrete variable it is written as a two-row table. There is one validity condition: the probabilities must be non-negative and add up to exactly one, because they exhaust every possibility.

How does expected value differ from a sample mean?

The expected value is a property of the model and is computed from the distribution — from the probabilities we assume. The arithmetic mean is a property of the data and is computed from what actually came up. The first is single and fixed, the second changes from sample to sample; over many repetitions the mean approaches the expected value, but it does not equal it.

Must the expected value be one of the possible values?

No, and that often surprises people. The expected number of pips on a die is 3.5, though a die never shows 3.5. It is the centre of mass of the distribution, not a result to expect from a single trial.

What is the variance of a random variable for?

For measuring how far the values scatter around E(X). Two variables can share an expected value and carry completely different risk — a game always paying 5 and a game paying 0 or 10 with equal chances both have E(X) = 5 but variances of 0 and 25. The standard deviation, the square root of the variance, expresses that scatter in the same unit as the variable itself.

Why is the variance computed as E(X²) − E(X)²?

Because it needs one pass over the table instead of two. The definitional formula needs E(X) first and then a second pass over every deviation. The computational version accumulates both sums at once and gives the same answer — it is an identity, not an approximation.

What is the distribution function?

The function F(x) = P(X ≤ x), the probability that the variable does not exceed a given number. For a discrete variable it is a step function: constant between successive values and jumping by p at each of them. It is always non-decreasing, tends to 0 at minus infinity and to 1 at plus infinity.

How does a continuous variable differ from a discrete one?

By the set of values. A discrete variable takes values that can be listed, and each has positive probability. A continuous one — a waiting time, a height — takes every value in an interval, and the probability of hitting any particular number is zero. The sum is then replaced by an integral of a density function, and the probability of an interval is the area under its graph.

Advanced level

Random variables and expected value

A random variable attaches a number to every outcome of an experiment — a payout, a waiting time, a count of faults. Its distribution is written as a table, and two numbers summarise that whole table: the expected value says what the results centre on, the variance says how far from it they wander.

Before you start

This topic builds on earlier ideas. Before you start, it's worth working through the lessons below — they'll make everything click:

Trees and the Bernoulli schemeA multi-stage experiment draws as a tree: probabilities multiply along the branches and add across the paths. That one picture holds the formula for the union of two events, the difference between disjoint and independent events, and the whole Bernoulli scheme — including where its binomial coefficient comes from.
Mean, median and modeThree answers to one question: which single number best describes a whole data set. The mean shares the total out evenly, the median stands exactly in the middle, and the mode points at the most frequent value — and which one you pick can change the conclusion.
Variance and deviationTwo data sets can share the same mean and look nothing alike. The range, the variance and the standard deviation measure what the mean cannot see: how widely the data is spread around it.

Where this is used

Real situations where you count exactly the way this lesson teaches:

A lottery ticket
A ticket costs 5. The top prize of 1000 comes up with probability 1/2000 and a consolation 50 with probability 1/200. The expected payout is 1000/2000 + 50/200 = 0.50 + 0.25 = 0.75, so the player’s expected result is 0.75 − 5 = −4.25 per ticket. A hundred tickets a year means an expected loss of 425, and no run of luck changes that — it only changes when it shows.
Pricing an insurance premium
An insurer estimates that a claim occurs for 2 per cent of customers per year and that its average cost is 30,000. The expected payout per policy is 0.02 · 30,000 = 600, so a premium of 900 leaves 300 for costs and profit. The whole actuarial pricing of a policy is the computation of the expected value of the random variable “payout”.
European roulette
A stake of 1 on a single number pays 35 to 1, and there are 37 numbers counting the zero. The expected value is 35 · 1/37 − 1 · 36/37 = −1/37, about −0.027 per unit staked. That is the house edge: 2.7 per cent of turnover, whatever the betting strategy and however many times the game is played.
A flat-rate service contract
A repair shop knows that a given machine breaks down within a year with probability 0.15 and that a repair costs 800. The expected cost is 0.15 · 800 = 120 per machine, so a flat annual fee of 200 covers it with room to spare. It is worth taking for the customer whenever they prefer a certain 200 to a lottery that costs 800 with probability 15 per cent — and that is the whole difference between expected value and risk.

All formulas

Random variable
$X: \Omega \to \mathbb{R}$
a function attaching a number to an outcome
Distribution
$\sum_{i} p_i = 1, \qquad p_i = P(X = x_i)$
a table of values and their probabilities
Expected value
$E(X) = \sum_{i} x_i \, p_i$
a mean weighted by probabilities
Variance of a variable
$\sigma^2(X) = \sum_{i} \big(x_i - E(X)\big)^2 p_i$
mean squared deviation from E(X)
Computational form
$\sigma^2(X) = E(X^2) - \big(E(X)\big)^2$
one pass over the table instead of two
Standard deviation
$\sigma(X) = \sqrt{\sigma^2(X)}$
in the same unit as X
Distribution function
$F(x) = P(X \leq x)$
non-decreasing, from 0 to 1
Linear transformation
$E(aX + b) = a\,E(X) + b, \qquad \sigma^2(aX + b) = a^2 \sigma^2(X)$
a shift does not change the spread

The outcome of rolling two dice is a pair of faces, but a problem almost always asks about their sum. The outcome of a game is the whole play, but what interests us is the payout. In both cases an event is translated into a number — and that recipe for translating is what a random variable is.

A random variable is a function

A random variable is a function attaching a number to every elementary outcome:

X: \Omega \to \mathbb{R}

The name misleads twice. It is not a variable in the algebraic sense — it does not stand in an equation and there is nothing to solve for. Nor is it a random number — the function is entirely determined; only its argument is random.

For a roll of two dice $\Omega$ has $36$ elements, and $X$ = the sum of the faces assigns the number $7$ to the pair $(3, 4)$ . The variable “simplifies” the experiment: instead of $36$ pairs we have $11$ numbers — at the price that they are no longer equally likely.

The distribution as a table

The distribution of a discrete variable is the list of its values together with their probabilities. It is written as a table:

$x_i$	$0$	$5$	$10$
$p_i$	$\tfrac{1}{4}$	$\tfrac{1}{2}$	$\tfrac{1}{4}$

There is one validity condition, and it is always worth checking:

p_i \geq 0, \qquad \sum_{i} p_i = 1

The table exhausts every possibility, so the probabilities must add up to exactly one. The same distribution drawn as bars:

The distribution of the payout in game A. The bar heights add up to one — it is the same table, only drawn.

Expected value

The expected value of a random variable is the mean of its values weighted by their probabilities:

E(X) = \sum_{i} x_i \, p_i

Physically it is the centre of mass of the distribution: the point where the bar chart could be balanced on a fingertip. For game A:

E(X) = 0 \cdot \frac{1}{4} + 5 \cdot \frac{1}{2} + 10 \cdot \frac{1}{4} = 0 + 2.5 + 2.5 = 5

Two remarks, both more important than the arithmetic itself.

The expected value need not be possible. For a die roll $E(X) = \tfrac{1+2+\dots+6}{6} = 3.5$ , though a die never shows $3.5$ . It is a centre of mass, not a forecast of a single result.

The expected value is not a sample mean. The arithmetic mean is computed from what came up and changes from sample to sample. The expected value is computed from the model, that is from the probabilities, and is single. After a thousand die rolls the mean will be close to $3.5$ ; after ten it may well be $2.8$ — and $E(X)$ is still $3.5$ in both cases.

A lottery ticket costs 5. A prize of 1000 occurs with probability 1/2000 and a prize of 50 with probability 1/200. Is it worth playing?

Variance: what the expected value does not say

$E(X)$ on its own summarises a distribution too aggressively. Compare game A with game B, whose payout is $4$ , $5$ or $6$ with the same probabilities:

Game B on the same scale as game A. The same expected value of 5, but the bars huddle around the centre instead of spreading across the whole axis.

Both games have $E(X) = 5$ and differ in everything that matters to a player. That difference is measured by the variance of the variable:

\sigma^2(X) = \sum_{i} \big(x_i - E(X)\big)^2 p_i

In practice the computational formula is handier, needing one pass over the table instead of two:

\sigma^2(X) = E(X^2) - \big(E(X)\big)^2

Compute the variance and standard deviation for both games.

Mind the units: the variance is in squared units, so what compares with the variable itself is the standard deviation $\sigma(X)$ . It is the same relationship that appeared for the spread of a data set — with the difference that there the weights were counts, and here they are probabilities.

Two properties of linear transformations are also worth having:

E(aX + b) = a\,E(X) + b, \qquad \sigma^2(aX + b) = a^2 \sigma^2(X)

A shift by $b$ does not change the spread — it moves the whole distribution rather than stretching it. That is why $b$ does not appear in the second formula at all.

The distribution function

Instead of asking “what is $X$ ”, we often ask “will $X$ stay below a given value”. That is answered by the distribution function:

F(x) = P(X \leq x)

For a discrete variable it is a step function: constant between the values, jumping by $p_i$ at each of them. For game A:

interval	$x < 0$	$0 \leq x < 5$	$5 \leq x < 10$	$x \geq 10$
$F(x)$	$0$	$\tfrac{1}{4}$	$\tfrac{3}{4}$	$1$

The distribution function is always non-decreasing, starts at $0$ and ends at $1$ — because probabilities accumulate rather than cancel. Its convenience is that the probability of an interval is read off by subtraction: $P(a < X \leq b) = F(b) - F(a)$ .

A note on continuous variables

Everything above assumed the values could be listed. Sometimes they cannot: a bus waiting time, the height of a randomly chosen person or a measurement error take every value in an interval.

For such a continuous variable the probability of hitting any particular number is zero — not because it is impossible, but because there are infinitely many possibilities. The sensible question is about an interval, and it is answered by the area under the graph of a density function:

P(a \leq X \leq b) = \int_a^b f(x)\,dx

The sum over a table turns into a definite integral, and the condition “the probabilities add up to one” turns into “the area under the whole density is one”. The most important continuous variable of all, the normal distribution, comes two lessons further on.

Exercises

The set asks for two numbers, both computed from the same distribution table. The $E(X)$ prompt gives “value → probability” pairs and asks for the expected value. The $\sigma^2(X)$ prompt gives a table of the same shape and asks for the variance of the variable, that is for $E(X^2) - \big(E(X)\big)^2$ .

The answer here is a number, not a fraction in lowest terms: an expected value is not a probability but a quantity on the variable’s own scale. The probabilities in the tables are chosen so that the result has at most two decimal places.

Practice

Work through a set of exercises — they get harder as you go. At the end you'll see your score and the mistakes worth reviewing.

Exercise 1 of 8Score: 0

E(X): 0 → 3/5, 2 → 1/5, 3 → 1/5

The generator does not ask you to state a distribution — the answer would be a whole table together with the names of the events, not a number. Nor does it ask about the distribution function, for the same reason. Both are worked through in the examples above.

Common mistakes

Confusing $E(X)$ with the arithmetic mean of data — the first is a property of the model and is fixed, the second a property of the sample and changes with every repetition.
Expecting $E(X)$ to be one of the possible values — $3.5$ pips never comes up, and it is still the correct expected value.
A table whose probabilities do not add up to one — that is not a distribution, and every quantity computed from it is meaningless. It is worth starting with that check.
Squaring the probabilities instead of the values — in $E(X^2)$ the squares apply to the $x_i$ , while the weights $p_i$ stay untouched.
Comparing a variance with the variable itself — a variance is in squared units; what compares is the standard deviation.
Concluding that two variables with the same $E(X)$ are equivalent — games A and B share an expected value and differ twenty-five-fold in variance, which for a player is the whole difference.

Formula card

Topic: Random variables

Random variable
$X: \Omega \to \mathbb{R}$
a function attaching a number to an outcome
Distribution
$\sum_{i} p_i = 1, \qquad p_i = P(X = x_i)$
a table of values and their probabilities
Expected value
$E(X) = \sum_{i} x_i \, p_i$
a mean weighted by probabilities
Variance of a variable
$\sigma^2(X) = \sum_{i} \big(x_i - E(X)\big)^2 p_i$
mean squared deviation from E(X)
Computational form
$\sigma^2(X) = E(X^2) - \big(E(X)\big)^2$
one pass over the table instead of two
Standard deviation
$\sigma(X) = \sqrt{\sigma^2(X)}$
in the same unit as X
Distribution function
$F(x) = P(X \leq x)$
non-decreasing, from 0 to 1
Linear transformation
$E(aX + b) = a\,E(X) + b, \qquad \sigma^2(aX + b) = a^2 \sigma^2(X)$
a shift does not change the spread

The distribution of the payout in game A: 0, 5 or 10 units with probabilities 1/4, 1/2 and 1/4. The bars add up to one, and the centre of mass of this table — the expected value — sits at 5.

The distribution of the payout in game B: 4, 5 or 6 with the same probabilities, drawn to the same scale. The same expected value of 5, a variance twenty-five times smaller.

Frequently asked questions

Luminous histogram bars settling into the soft bell of a distribution, one outlying point brighter than the rest, against a cosmic nebula in violet and magenta

Statistics and probability

The average person does not exist

All of descriptive statistics rests on a single trade: you give up information about a crowd and receive one number in return. The trade is often worth making, but it is never free — and it pays to know exactly what gets lost in it. From the mortality tables of 1662, through Quetelet's average man, to the aircraft nobody ever saw.

16 min