Law of Large Numbers Statement
In probability notation:
Equivalently (Weak Law form): for any ε > 0, P(|X̄ₙ − μ| ≥ ε) → 0 as n → ∞. The sample mean converges in probability to the true population mean.
What Is the Law of Large Numbers?
In concrete terms: run any random experiment enough times — rolling a die, sampling household incomes, measuring machine output — and the average of your results will get arbitrarily close to the theoretical average that probability assigns to that process. The longer you run the experiment, the closer the match.
This idea was first proved rigorously by Jacob Bernoulli in 1713 for binary outcomes in Ars Conjectandi. Siméon Denis Poisson generalized it and coined loi des grands nombres in 1837. Pafnuty Chebyshev provided a clean algebraic proof in 1867, and Andrey Kolmogorov established the modern measure-theoretic foundation in 1933.
The LLN sits at the foundation of hypothesis testing, confidence intervals, and virtually every form of statistical inference. It is the mathematical justification for one of science's most intuitive beliefs: more data gives a more accurate picture of reality. Return to Statistics Fundamentals to explore the full range of related topics.
- Abbreviation: LLN (in statistics and probability)
- Core claim: Sample mean X̄ₙ converges to population mean μ as n → ∞
- Two forms: Weak law (convergence in probability) and strong law (almost sure convergence)
- Requires: Independent, identically distributed random variables with finite expected value
- Variance formula: Var(X̄ₙ) = σ²/n — variance of the sample mean shrinks as n grows
- Does not mean: Individual outcomes are predicted or corrected by the law (Gambler's Fallacy)
- Other names: Law of big numbers, large number theorem, Bernoulli's theorem
Law of Large Numbers Formula
Let X₁, X₂, …, Xₙ be a sequence of independent and identically distributed random variables, each with a finite expected value E[Xᵢ] = μ. The Law of Large Numbers formula for the sample mean is:
X̄ₙ = sample mean of n observations
n = number of trials
Xᵢ = i-th observation
μ = true population mean E[X]
The LLN asserts that as n → ∞, this quantity X̄ₙ approaches μ. The exact sense in which it "approaches" μ differs between the weak and strong forms, covered below.
The Mathematics of Variance Reduction
The structural reason the law works is visible in how variance behaves. If the population distribution has variance σ², then the variance of the sample mean is:
σ² = population variance
n = sample size
Because n sits in the denominator, the variance of the sample mean decreases toward zero as sample size grows. At the limit:
limn→∞ Var(X̄ₙ) = limn→∞ σ²/n = 0 | The probability distribution of the sample mean compresses into a sharp spike centered exactly at μ. This is why large samples are more reliable than small ones — not because of luck, but because of mathematics.
This relationship connects directly to the sampling distribution of the sample mean and explains why the Central Limit Theorem uses σ/√n as the standard error.
Weak Law vs. Strong Law of Large Numbers
The Law of Large Numbers comes in two mathematically distinct forms. Both reach the same conclusion — that X̄ₙ approaches μ — but they differ in what "approaches" means precisely.
Weak Law of Large Numbers (WLLN)
For any arbitrarily small ε > 0:
Interpretation: As n grows, the probability that the sample mean deviates from the true mean by more than any fixed amount ε goes to zero. It guarantees convergence in probability.
The weak law says that for any specific value of n, it is overwhelmingly likely (but not guaranteed across all realizations) that the sample mean stays close to μ. Chebyshev's Inequality provides a direct algebraic proof: P(|X̄ₙ − μ| ≥ ε) ≤ σ²/(nε²), and as n → ∞ the right side goes to zero. See the proof walkthrough in the Chebyshev proof section below.
Strong Law of Large Numbers (SLLN)
Interpretation: The sample mean converges to the true mean almost surely — with probability 1 — for every possible infinite sequence of outcomes. It guarantees almost sure convergence.
Kolmogorov proved this stronger result in 1933 using measure theory. The strong law makes the WLLN's guarantee permanent: once n is large enough, the path of the running average locks onto μ and does not wander away. This is the form most used in mathematical statistics and econometrics.
| Feature | Weak Law (WLLN) | Strong Law (SLLN) |
|---|---|---|
| Type of convergence | Convergence in probability | Almost sure convergence |
| Probability statement | P(|X̄ₙ − μ| ≥ ε) → 0 | P(lim X̄ₙ = μ) = 1 |
| Isolated extreme deviations | Allowed (probability → 0) | Ruled out (probability exactly 0) |
| Proved by | Chebyshev / Khinchin (1867/1929) | Kolmogorov (1933) |
| Tools required | Chebyshev's Inequality | Measure theory / Borel-Cantelli |
| Practical difference | Almost none in applications | Stronger theoretical guarantee |
Law of Large Numbers vs. Central Limit Theorem
The Law of Large Numbers (LLN) and the Central Limit Theorem (CLT) are frequently confused because both describe what happens to sample means as n grows. They answer different questions.
The LLN says the sample mean will equal μ in the limit. The CLT says the distribution of the sample mean approaches a normal curve. These are complementary facts about different aspects of sampling behavior. See the Central Limit Theorem guide for a full treatment.
| Comparison Point | Law of Large Numbers | Central Limit Theorem |
|---|---|---|
| Primary question | What value does X̄ₙ approach? | What shape is the distribution of X̄ₙ? |
| End result | A single point: X̄ₙ → μ | A distribution: X̄ₙ ≈ N(μ, σ²/n) |
| What it tracks | Convergence of the mean value | Convergence of the error distribution |
| Variance in limit | Var(X̄ₙ) → 0 | SE = σ/√n (normalizes the spread) |
| Requires normality? | No | No (but result is normal shape) |
| Practical use | Justifies using X̄ as estimate of μ | Builds confidence intervals and z-tests |
The two theorems work together. The LLN guarantees your sample mean will land near μ. The CLT tells you how to calculate the probability it lands within any specific range of μ — which is exactly what confidence intervals for the mean and hypothesis tests compute.
How the Law of Large Numbers Works: Three Phases
Convergence progresses through three recognizable phases regardless of the underlying distribution.
Volatile Micro-Sample (n ≤ 10)
Individual outcomes dominate the average. A fair coin flipped 4 times landing on heads 3 times (75%) is common. Do not draw conclusions from small samples.
Stabilizing Meso-Sample (10 < n < 1,000)
Random runs lose their mathematical leverage. The running average visibly moves toward the expected value line. Variance has dropped by a factor of 100 compared to n = 1.
Converged Macro-Sample (n ≥ 1,000)
The sample mean locks tightly onto μ. At n = 10,000 coin flips, the deviation from exactly 50% heads is typically under 1%. This is the regime in which insurance pricing and casino mathematics operate.
Characteristics of the Law of Large Numbers
Which of these statements is NOT a characteristic of the Law of Large Numbers?
- ✓ TRUE: As n increases, the sample mean X̄ₙ converges to the population mean μ
- ✓ TRUE: The variance of the sample mean decreases as Var(X̄ₙ) = σ²/n
- ✓ TRUE: It requires independent, identically distributed (i.i.d.) random variables
- ✓ TRUE: It applies in both weak (in probability) and strong (almost sure) forms
- ✗ NOT TRUE: "Future outcomes will correct past deviations to restore the average" — this is the Gambler's Fallacy, NOT the LLN
- ✗ NOT TRUE: "It predicts or guarantees any individual outcome" — the LLN says nothing about individual future results
- ✗ NOT TRUE: "It applies to small samples" — the LLN applies only as n grows large toward infinity
Worked Examples — Law of Large Numbers in Action
Example 1: The Coin Toss (Binary Uniform Distribution)
A fair coin assigns X = 1 for heads and X = 0 for tails. The theoretical expected value is E[X] = 0.5. What happens to the sample mean across growing trial counts?
Setup: E[X] = (1 × 0.5) + (0 × 0.5) = 0.5. Variance σ² = 0.25. Var(X̄ₙ) = 0.25/n.
Small sample (n = 10): 7 heads, 3 tails. X̄₁₀ = 0.70. Deviation from μ: +0.20. Var(X̄₁₀) = 0.025.
Medium sample (n = 100): 53 heads. X̄₁₀₀ = 0.53. Deviation: +0.03. Var(X̄₁₀₀) = 0.0025.
Large sample (n = 10,000): 5,004 heads. X̄₁₀,₀₀₀ = 0.5004. Deviation: +0.0004. Var = 0.000025.
| Trials (n) | Heads | Sample Mean X̄ₙ | Deviation from 0.5 | Var(X̄ₙ) |
|---|---|---|---|---|
| 10 | 7 | 0.7000 | +0.2000 | 0.02500 |
| 100 | 53 | 0.5300 | +0.0300 | 0.00250 |
| 1,000 | 507 | 0.5070 | +0.0070 | 0.00025 |
| 10,000 | 5,004 | 0.5004 | +0.0004 | 0.000025 |
✅ Each tenfold increase in sample size reduces the deviation from μ by roughly a factor of three (1/√10 ≈ 0.316). The sample mean converges to 0.5 as the Law of Large Numbers predicts.
Example 2: The Six-Sided Die
Rolling a fair die: outcomes X ∈ {1,2,3,4,5,6} each with probability 1/6. Expected value μ = 3.5.
Expected value: μ = (1+2+3+4+5+6)/6 = 3.5. Variance σ² = 2.917.
Small sample (n = 6): Rolls: {1,1,4,6,2,5}. X̄₆ = 3.167. Deviation: −0.333.
Large sample (n = 60,000): Frequencies approach 1/6 per face. X̄₆₀,₀₀₀ ≈ 3.5007. Deviation: +0.0007.
✅ The sample mean converges to 3.5. The mechanism: Var(X̄ₙ) = 2.917/n → 0 as n → ∞.
Law of Large Numbers in Insurance
The Law of Large Numbers in insurance is the actuarial principle that enables stable premium pricing. By pooling a large number of policyholders, an insurer ensures that actual claims closely match statistical projections.
An insurer covers policyholders with a 1% annual probability of a $100,000 claim. Expected payout per policyholder: E[X] = $1,000. How does pool size affect pricing stability?
Small pool (n = 100): Expected claims: 1 at $100,000. If 3 claims occur by chance, actual cost = $300,000 — a 200% overrun. Per-policyholder cost spikes to $3,000 against a $1,000 premium. The variance per policyholder is enormous relative to pool size.
Large pool (n = 1,000,000): Expected claims: 10,000 × $100,000 = $1,000,000,000 total. Actual claims fall between 9,900 and 10,100 with very high probability. Per-policyholder cost stabilizes to within a few dollars of $1,000.
The math: σ² per policyholder ≈ $990,000,000. At n = 1,000,000: Var(X̄) = σ²/n ≈ $990. Standard deviation of average payout ≈ $31.46 — negligible against a $1,000 premium.
✅ The LLN enables insurers to set stable premiums. With millions of policyholders, actual per-person payout converges so closely to expected value that the company can price risk accurately and remain solvent.
According to the Law of Large Numbers, how would losses be affected by pooling more insureds? Pooling more insureds reduces the variability of per-capita losses. As n increases, Var(X̄ₙ) = σ²/n decreases, making the actual average loss more predictable and closer to the expected value. Larger pools lead to more stable and accurate pricing — this is why large insurers can offer lower premiums per unit of coverage than small ones.
LLN vs. the Gambler's Fallacy
The Gambler's Fallacy is the most widespread misreading of the Law of Large Numbers. It holds that if random outcomes have deviated from their expected pattern in the short term, they must "correct" themselves in the near future.
"A coin has come up heads 8 times in a row. Tails is now much more likely on the next flip." — This reasoning is false. The coin has no memory. The probability of heads on flip 9 remains exactly 0.5.
The LLN does not work by forcing future outcomes to compensate for past deviations. It works by diluting past anomalies with a large volume of new, ordinary results.
| Scenario | Gambler's Fallacy (Wrong) | Law of Large Numbers (Correct) |
|---|---|---|
| 8 heads in a row | Tails is "due." Next flip biased toward tails. | Next flip is still 50/50. Independence is absolute. |
| Mechanism | Outcomes self-correct to restore the average. | Anomaly is diluted by thousands of subsequent ordinary flips. |
| 10,008 total flips (8 + 10,000 new) | 10,000 new flips should favor tails to compensate. | 10,000 new flips land ~5,000 each. Ratio: 5,008/10,008 = 50.04%. Converged. |
See the basic probability page for more on independence and the probability rules that govern individual events.
Real-World Applications of the Law of Large Numbers
Insurance & Actuarial Science
Actuaries pool millions of policyholders so actual claims closely match projections. See the expected value guide for premium calculation details.
Casino Operations
American roulette has a 5.26% house edge. Across millions of spins, actual revenue converges precisely to that margin. Short-term player wins do not threaten long-run profitability.
Polling & Survey Research
Larger samples produce smaller margins of error — a direct consequence of Var(X̄ₙ) = σ²/n. Related: confidence intervals.
Quantitative Finance
Monte Carlo pricing models run millions of simulated asset paths. The LLN ensures the average converges to the true theoretical option value. See Value at Risk explained.
Machine Learning
Empirical Risk Minimization trains models on finite datasets. The LLN guarantees that training error on a large dataset approximates true generalization error — why deep learning requires massive datasets.
Clinical Trials
Large trial sample sizes ensure observed treatment effects converge to true population effects. Power analysis is a direct application of the LLN and CLT. Related: power of a test.
Law of Large Numbers and the Monte Carlo Method
The Monte Carlo method is one of the most important practical applications of the LLN. It uses repeated random sampling to obtain numerical results — for example, approximating integrals or option prices that are difficult to compute analytically.
The algorithm works by: (1) generating n independent uniform random samples, (2) evaluating the function at each sample point, and (3) averaging the results. By the LLN, the sample average converges to the true integral value as n → ∞. With 25 samples you get a rough approximation; with 25,000 samples, the result is highly accurate. This is the LLN in direct computational action.
LLN for Chi-Square: Why Xₙ/n → 1
A frequently searched application of the LLN is its use with Chi-square random variables. If Xₙ follows a Chi-square distribution with n degrees of freedom, then by the Law of Large Numbers:
Why? A Chi-square(n) random variable is the sum of n independent squared standard normal variables, so E[Xₙ] = n. Dividing by n gives a sample mean of n variables each with expected value 1. By the LLN, this sample mean converges to 1. This result is used in asymptotic theory for goodness-of-fit tests and likelihood ratio tests. For related tools, see the Chi-square test guide and chi-square table.
Borel's Law of Large Numbers
Borel's law of large numbers, named after Émile Borel, states that if an experiment is repeated a large number of times independently under identical conditions, the proportion of times any specified event occurs will converge to that event's probability. Formally, if E is an event with probability p and Nₙ(E) is the number of occurrences of E in n trials:
This is a special case of the strong LLN applied to Bernoulli random variables, and it makes rigorous the intuitive notion of probability as the long-run relative frequency of an event.
Uniform Law of Large Numbers
The Uniform Law of Large Numbers (ULLN) extends the LLN from a single function to a class of functions. It states that empirical averages converge to expected values uniformly over a class of functions — not just for one fixed function. The foundational result is the Glivenko-Cantelli theorem, which proves that the empirical CDF converges uniformly to the true CDF. The ULLN underpins consistency proofs in machine learning, including proving that empirical risk minimization converges to the true risk. Related to statistics for machine learning.
Interactive LLN Convergence Simulator
Watch the sample mean converge toward the true expected value in real time. Select an experiment and trial count, then run the simulation.
Law of Large Numbers — Convergence Simulator
Proof via Chebyshev's Inequality (Weak Law)
The weak law has a clean, accessible proof using Chebyshev's Inequality that requires only knowledge of expected value and variance.
Apply to the Sample Mean
Set Y = X̄ₙ. We know E[X̄ₙ] = μ and Var(X̄ₙ) = σ²/n. Substitute into Chebyshev's Inequality.
Obtain the Probability Bound
P( |X̄ₙ − μ| ≥ ε ) ≤ σ²/n / ε² = σ² / (nε²). For any fixed ε > 0 and σ², this bound depends on n in the denominator.
Take the Limit
As n → ∞: σ² / (nε²) → 0. Therefore P( |X̄ₙ − μ| ≥ ε ) → 0 for every ε > 0. This is the Weak Law of Large Numbers.
Three steps, one inequality. The weak law follows directly from Var(X̄ₙ) = σ²/n → 0. See also: variance and standard deviation.
Related Concepts and Resources
| Concept | Connection to LLN | Resource |
|---|---|---|
| Expected Value | The quantity X̄ₙ converges toward — the target of the LLN | Expected Value Guide |
| Central Limit Theorem | Describes the distribution shape of X̄ₙ as n grows | CLT Guide |
| Sampling Distributions | The sampling distribution of X̄ compresses toward μ per LLN | Sample Mean Distribution |
| Confidence Intervals | Require X̄ₙ to estimate μ reliably — valid because of the LLN | CI for Mean |
| Hypothesis Testing | Uses X̄ₙ as an estimator — valid because of the LLN | Hypothesis Testing Hub |
| Inferential Statistics | LLN is the theoretical justification for all inference from samples | Inferential Statistics |
| Basic Probability | Independence of trials is the core requirement of the LLN | Basic Probability |
| Bayes Theorem | LLN connects frequentist and Bayesian views of probability convergence | Bayes' Theorem |
| Bayesian vs. Frequentist | The LLN justifies the frequentist interpretation of probability | Bayesian vs. Frequentist |
| Normal Distribution | Shape the distribution of X̄ₙ converges to via CLT | Normal Distribution |
| Sample Size Calculator | Determines n needed for desired precision — driven by LLN logic | Sample Size Calculator |
| Correlation vs. Causation | LLN underlies why large samples narrow causal inference errors | Correlation vs. Causation |