Probabilistic foundations of Empirical CDF & DKW Bound.
The Empirical Cumulative Distribution Function (ECDF) is the fundamental non-parametric representation of a sample dataset, serving as the empirical analogue of the population distribution function. The celebrated Glivenko-Cantelli theorem proves that the ECDF converges uniformly to the true distribution function as sample size increases. Furthermore, the Dvoretzky–Kiefer–Wolfowitz (DKW) inequality provides finite-sample probability bounds that enable the construction of simultaneous distribution-free confidence envelopes. In this laboratory, we construct empirical distribution functions, compute supremum distance metrics, and confirm the universal N^(-1/2) convergence law governing simultaneous confidence bands. In mathematical statistics, empirical data science, and modern computational engineering, understanding the asymptotic sampling properties and theoretical behavior of Empirical Cumulative Distribution Function and Dvoretzky–Kiefer–Wolfowitz Inequality provides crucial guarantees. Rigorous experimental validation connects foundational probability laws with observable empirical phenomena, ensuring that statistical inference remains valid under finite sample sizes and real-world data collection constraints.
Viva voce questions & analytical theory
What is the formal definition of the Empirical Cumulative Distribution Function (ECDF)?
Let X_1, ..., X_N be i.i.d. random variables with CDF F(x). The ECDF F_N(x) is defined as F_N(x) = (1 / N) ∑_{i=1}^N I(X_i ≤ x). For any fixed x, N · F_N(x) is a binomial variable ~ Binomial(N, F(x)). Hence E[F_N(x)] = F(x) and Var(F_N(x)) = F(x)(1 - F(x)) / N.
What is the Glivenko-Cantelli Theorem, and why is it called the Fundamental Theorem of Statistics?
The Glivenko-Cantelli theorem (1933) proves uniform almost sure convergence: sup_{x ∈ ℝ} |F_N(x) - F(x)| → 0 as N → ∞ with probability 1. Unlike pointwise LLN, it guarantees uniform convergence everywhere simultaneously, showing an empirical sample completely determines the population distribution asymptotically.
What does the Dvoretzky–Kiefer–Wolfowitz (DKW) inequality state, and what was Massart's contribution?
The DKW inequality bounds the probability of uniform deviation: P(sup_x |F_N(x) - F(x)| > ε) ≤ C exp(-2Nε^2). Pascal Massart (1990) proved the tight constant C = 2 for all ε > 0 with 2 exp(-2Nε^2) ≤ 1. Setting this to α yields simultaneous confidence bands F_N(x) ± √[ln(2/α) / (2N)].
How do simultaneous DKW confidence bands differ from pointwise binomial intervals?
Pointwise confidence intervals guarantee 1 - α coverage at a single specified x. Simultaneous DKW bands guarantee the entire population curve F(x) remains inside the band across all x ∈ ℝ simultaneously with probability at least 1 - α.
How is the Kolmogorov-Smirnov one-sample goodness-of-fit test formulated?
The test statistic is the empirical supremum deviation D_N = sup_x |F_N(x) - F_0(x)|. Kolmogorov (1933) showed √N D_N converges in distribution to the supremum of a standard Brownian bridge on [0, 1], with limit CDF K(z) = 1 - 2 ∑_{k=1}^∞ (-1)^(k-1) exp(-2k^2 z^2).
Why is the Kolmogorov-Smirnov test statistic distribution-free for continuous random variables?
By the probability integral transform, U = F_0(X) is standard uniform U[0, 1] for any continuous CDF F_0. Transforming X_i to U_i maps D_N to sup_{u ∈ [0, 1]} |G_N(u) - u|, whose null distribution depends solely on sample size N and is independent of F_0.
What is the Brownian bridge, and how does it arise in empirical process theory?
The normalized empirical process α_N(t) = √N (F_N(F^(-1)(t)) - t) converges weakly in Skorokhod space to a standard Brownian bridge B(t) = W(t) - t W(1), a zero-mean Gaussian process tied down to zero at endpoints t = 0 and t = 1 with covariance min(s, t) - st.
Experimental precautions and statistical checks
- Ensure data has no tied values when applying exact continuous Kolmogorov-Smirnov test formulas.
- Remember that standard DKW bounds assume target distribution parameters are known a priori; estimating parameters from data alters the critical values.
- Truncate simultaneous band boundaries at [0, 1] because probabilities outside this unit interval are mathematically impossible.
- Perform replication across multiple independent pseudorandom seeds to verify that estimator variance and bias bounds are invariant to initial generator states.
Theoretical assumptions & limitations
- DKW confidence bands are conservative in the distribution tails where F(x)(1 - F(x)) approaches zero, where local variance is much smaller than 1/4.
- For discrete random variables with probability atoms, the distribution-free property fails and critical values must be adjusted downward.