Law of Large Numbers theory

The Weak Law of Large Numbers (Khinchin's theorem) states that if X_1, X_2, ... are i.i.d. random variables with finite mean μ and variance σ^2 < ∞, then for any ε > 0, lim_{n→∞} P(|X̄_n - μ| ≥ ε) = 0. This is convergence in probability (X̄_n →^P μ). Proof: E[X̄_n] = μ and Var(X̄_n) = σ^2 / n. Applying Chebyshev's inequality: P(|X̄_n - μ| ≥ ε) ≤ Var(X̄_n) / ε^2 = σ^2 / (n ε^2). As n → ∞, σ^2 / (n ε^2) → 0, establishing that the probability of any deviation strictly vanishes.

The Strong Law of Large Numbers (Kolmogorov's theorem) establishes almost sure convergence: P(lim_{n→∞} X̄_n = μ) = 1. While the Weak Law states that for any given large n, the probability of finding a deviation is small (pointwise in time), the Strong Law guarantees that the entire sample path trajectory X̄_n stays confined within (μ - ε, μ + ε) for all n beyond some random index N_0 with probability 1. Almost sure convergence is a pathwise topological property and is strictly stronger than convergence in probability.

Andrey Kolmogorov (1930) proved that for independent random variables X_n with variances σ_n^2, if ∑_{n=1}^∞ (σ_n^2 / n^2) < ∞, then the Strong Law of Large Numbers holds: (S_n - E[S_n]) / n → 0 almost surely. For i.i.d. variables, Kolmogorov proved that the existence of a finite first moment E[|X|] < ∞ is both necessary and sufficient for the Strong Law of Large Numbers to hold, requiring no assumptions about finite variance.

The standard Cauchy distribution has probability density f(x) = 1 / [π(1 + x^2)]. The characteristic function of a Cauchy variable is φ_X(t) = exp(-|t|). For the sample mean X̄_n = (1/n) ∑ X_i, the characteristic function is φ_{X̄_n}(t) = [φ_X(t/n)]^n = [exp(-|t/n|)]^n = exp(-|t|). Because the characteristic function of the average of n Cauchy variables is identical to that of a single Cauchy variable, X̄_n has exactly the same Cauchy distribution for all n! As n increases, the sample mean does not concentrate around any constant; it continues to fluctuate wildly across the entire real line because the expected value integral diverges.

Alexander Khinchin (1924) established the Law of the Iterated Logarithm, which describes the precise boundary of oscillations of partial sums S_n: limsup_{n→∞} (S_n - nμ) / √(2 n σ^2 ln(ln(n))) = +1 and liminf_{n→∞} (S_n - nμ) / √(2 n σ^2 ln(ln(n))) = -1 almost surely. This proves that the sample mean X̄_n fluctuates around μ within an envelope of width ± σ √(2 ln(ln(n)) / n). The term √(ln(ln(n))) grows with extreme slowness, forming the exact mathematical barrier between the central limit theorem rate (O(1/√n)) and the strong law.

In Monte Carlo numerical integration, an unknown multidimensional integral I = ∫ g(x) dx is estimated as the sample average (1/N) ∑ g(X_i). The LLN guarantees that this empirical average converges to the true integral as N → ∞. In actuarial science and insurance, while individual claims are unpredictable random events, an insurer pooling N independent policies experiences an average payout per policy X̄_N that converges to the expected loss E[X], allowing insurers to set stable premiums with minimal risk of insolvency.

For any uncorrelated random variables with common variance σ^2: Var(X̄_N) = Var((1/N) ∑ X_i) = (1/N^2) ∑ Var(X_i) = (1/N^2) (N σ^2) = σ^2 / N. Taking the reciprocal defines estimation precision: Precision = 1 / Var(X̄_N) = (1 / σ^2) · N. When σ = 1, Precision = N, defining an exact linear relation y = 1.0000 · x with slope m = 1.0000 and intercept b = 0.0000.