Probabilistic foundations of Box Plots & Tukey's Fences.
Exploratory Data Analysis (EDA), pioneered by John W. Tukey, emphasizes visual and non-parametric techniques to uncover underlying data structures, detect anomalies, and summarize distributions without parametric assumptions. The box plot displays the five-number summary—minimum, first quartile, median, third quartile, and maximum—while utilizing Tukey's fences to isolate outliers. In this laboratory, we construct empirical box plots across controlled Gaussian distributions, analyze the properties of resistant summary statistics, and experimentally confirm the linear relation linking the interquartile range to the population standard deviation. In mathematical statistics, empirical data science, and modern computational engineering, understanding the asymptotic sampling properties and theoretical behavior of Exploratory Data Analysis, Five-Number Summary, and Tukey's Fences provides crucial guarantees. Rigorous experimental validation connects foundational probability laws with observable empirical phenomena, ensuring that statistical inference remains valid under finite sample sizes and real-world data collection constraints.
Viva voce questions & analytical theory
What is the five-number summary introduced by John W. Tukey?
Introduced in 1977 as part of Exploratory Data Analysis, the five-number summary comprises the sample minimum, Q1 (25th percentile), median Q2 (50th percentile), Q3 (75th percentile), and sample maximum. It provides a robust, distribution-free description of location and dispersion resistant to outliers.
How are Tukey's inner and outer fences defined mathematically?
With IQR = Q3 - Q1, the inner fences are Lower Inner Fence = Q1 - 1.5·IQR and Upper Inner Fence = Q3 + 1.5·IQR. Observations beyond inner fences are mild outliers. Outer fences are Q1 - 3·IQR and Q3 + 3·IQR, with points beyond them classified as extreme outliers.
What is the theoretical probability of an outlier in a Gaussian distribution under Tukey's 1.5·IQR rule?
For N(μ, σ^2), quartiles lie at μ ± 0.6745σ, giving IQR = 1.349σ. Inner fences lie at μ ± (0.6745 + 1.5 · 1.349)σ = μ ± 2.698σ. The two-sided tail probability is 2(1 - Φ(2.698)) ≈ 0.0070 (0.7%, or about 1 in 143 observations).
Why does the interquartile range scale as IQR ≈ 1.349 σ for normal distributions?
Since Φ((Q3 - μ)/σ) = 0.75, standard normal quantile Φ^(-1)(0.75) ≈ 0.67449. The interquartile range is IQR = Q3 - Q1 = 2 · Φ^(-1)(0.75) · σ ≈ 1.34898 · σ. This allows robust estimation of σ as IQR / 1.349.
How do box plot conventions handle discrete or tied data?
Hyndman and Fan classified 9 sample quantile types. For discrete data with ties, quantiles can coincide, causing the box or whiskers to collapse. In such cases, violin plots or jittered scatter points reveal distribution structure better.
What is the breakdown point of an estimator, and how does median compare to mean?
The breakdown point is the smallest fraction of corrupted observations that can drive an estimator to infinity. The sample mean has a 0% breakdown point (1 corrupted value ruins it). The sample median has a 50% breakdown point, and IQR has a 25% breakdown point.
How does skewness appear visually in a box plot?
In right-skewed data, the median lies closer to Q1 than Q3 ((Q3 - Q2) > (Q2 - Q1)), the upper whisker is longer than the lower whisker, and outlier markers appear predominantly above the upper fence.
Experimental precautions and statistical checks
- Do not discard outliers blindly without investigating whether they reflect genuine physical extremes.
- Use N ≥ 20 observations before interpreting box plot whiskers to avoid volatile quartile estimates.
- Maintain identical coordinate scaling across comparative box plot panels.
- Perform replication across multiple independent pseudorandom seeds to verify that estimator variance and bias bounds are invariant to initial generator states.
Theoretical assumptions & limitations
- Box plots cannot display multimodality; bimodal and uniform distributions can produce identical five-number summaries.
- Quantile values depend on the specific interpolation convention used in small datasets.