Probabilistic foundations of Single-Sample Z & t Tests.
Hypothesis testing is the primary inferential framework for scientific discovery and data-driven decision making. Codified by Jerzy Neyman and Egon Pearson, the methodology balances Type I false-positive risks against Type II false-negative errors through objective critical regions and p-values. In this laboratory, we implement one-sample Z-tests and Student's t-tests, simulate Type I and Type II error frequencies, investigate non-central distribution shifts, and confirm the non-centrality scaling gradient m = 2.000. In mathematical statistics, empirical data science, and modern computational engineering, understanding the asymptotic sampling properties and theoretical behavior of One-Sample Hypothesis Testing: Z-Tests, Student's t-Tests, Critical Values, and p-Values provides crucial guarantees. Rigorous experimental validation connects foundational probability laws with observable empirical phenomena, ensuring that statistical inference remains valid under finite sample sizes and real-world data collection constraints.
Viva voce questions & analytical theory
What is the Neyman-Pearson framework of statistical hypothesis testing?
Developed by Jerzy Neyman and Egon Pearson in 1933, the framework formalizes testing as a decision problem between two competing statistical hypotheses: the Null Hypothesis H_0 and Alternative Hypothesis H_1. Two decision errors can occur: (1) Type I Error (false positive): rejecting H_0 when H_0 is true, with probability P(Type I) = α (significance level); (2) Type II Error (false negative): failing to reject H_0 when H_1 is true, with probability P(Type II) = β. The statistical power is 1 - β, the probability of correctly rejecting a false null hypothesis.
What does the Neyman-Pearson Lemma prove regarding optimal decision rules?
The Neyman-Pearson Lemma is the foundational theorem of hypothesis testing. It states that for testing simple hypotheses H_0: θ = θ_0 versus H_1: θ = θ_1, the likelihood ratio test that rejects H_0 when Λ(x) = L(θ_0; x) / L(θ_1; x) ≤ k is the Uniformly Most Powerful (UMP) test of size α. That is, among all possible tests with Type I error probability at most α, the likelihood ratio test achieves the maximum possible power (minimum Type II error β).
What is the mathematical difference between a Z-test and a Student's t-test?
The choice between Z and t hinges on whether the population variance σ^2 is known or must be estimated: (1) One-sample Z-test: Used when population variance σ^2 is known a priori. The test statistic is Z = (x̄ - μ_0) / (σ / √n) ~ N(0, 1) under H_0. (2) One-sample t-test: Used when σ^2 is unknown and estimated by sample variance s^2 = (1/(n-1)) ∑ (X_i - x̄)^2. The statistic is T = (x̄ - μ_0) / (s / √n) ~ t(n - 1) under H_0. Because s introduces additional sampling variability, t critical values are strictly larger than Z critical values.
How is a p-value formally defined, and what are common misinterpretations?
The p-value is the probability, computed under the assumption that the null hypothesis H_0 is true, of obtaining a test statistic at least as extreme as the observed value: p = P(T ≥ t_obs | H_0). A small p-value (p ≤ α) indicates that the observed data is improbable under H_0, warranting rejection. Crucially, the p-value is NOT the probability that H_0 is true (a frequentist fallacy), nor is 1 - p the probability that H_1 is true; it is strictly a conditional statement about data given the null model.
What is the non-centrality parameter, and how does it determine statistical power?
Under the alternative hypothesis H_1: μ = μ_0 + Δ, the test statistic does not follow a central distribution, but rather a non-central distribution. For the Z-test, Z ~ N(λ, 1) where λ = (μ - μ_0) / (σ / √n) = δ √n is the non-centrality parameter (with δ = Δ/σ being Cohen's d effect size). The statistical power for an upper-tailed test of size α is Power = 1 - Φ(z_α - λ) = Φ(λ - z_α). Power increases monotonically with effect size δ and sample size √n.
How do one-tailed and two-tailed hypothesis tests compare in power and directionality?
A one-tailed (directional) test allocates the entire rejection region α to a single tail of the null distribution (e.g., critical value z_{0.05} = 1.645). A two-tailed (non-directional) test divides α equally between both tails (critical value z_{0.025} = 1.960). If the true effect lies in the hypothesized direction, a one-tailed test has strictly greater statistical power than a two-tailed test because its critical value is smaller; however, it has zero power to detect effects in the opposite direction.
What is the connection between hypothesis testing and confidence intervals?
Hypothesis testing and confidence intervals are mathematically dual formulations of statistical inference. A two-sided hypothesis test of size α rejects H_0: μ = μ_0 if and only if the hypothesized parameter value μ_0 falls outside the corresponding 100(1 - α)% confidence interval [x̄ - t_{α/2} s/√n, x̄ + t_{α/2} s/√n]. The confidence interval can be viewed as the set of all null hypotheses that would NOT be rejected by the data at significance level α.
Experimental precautions and statistical checks
- Formulate hypotheses and select significance level α a priori before examining experimental data to avoid p-hacking.
- Verify that population variance is genuinely known before using Z-tests; otherwise employ Student's t-test.
- Check that parent populations are approximately normal when conducting t-tests with small sample sizes (n < 30).
- Report effect sizes (Cohen's d) alongside p-values to distinguish practical significance from statistical significance.
Theoretical assumptions & limitations
- Statistical significance does not imply practical importance; very large sample sizes can yield tiny p-values for trivial effects.
- Failure to reject the null hypothesis H_0 does not prove that H_0 is true; it merely indicates insufficient evidence to reject it.