by Keith M. Bower, M.S. and James A. Colton, M.S.
Reprinted with permission from the American Society for Quality When performing statistical hypothesis tests such as a one-sample t-test or the AndersonDarling test for normality, an investigator will either reject or fail to reject the null hypothesis, based upon sampled data. Frequently, results in Six Sigma projects contain the verbiage “accept the null hypothesis,” which implies that the null hypothesis has been proven true. This article discusses why such a practice is incorrect, and why this issue is more than a matter of semantics.
Overview of Hypothesis Testing
In a statistical hypothesis test, two hypotheses are evaluated: the null (H0) and the alternative (H1). The null hypothesis is assumed true until proven otherwise. If the weight of evidence leads us to believe that the null hypothesis is highly unlikely (based upon probability theory), then we have a statistical basis upon which we may reject the null hypothesis.
A common misconception is that statistical hypothesis tests are designed to select the more likely of two hypotheses. Rather, a test will stay with the null hypothesis until enough evidence (data) appears to support the alternative.
The amount of evidence required to “prove” the alternative may be stated in terms of a confidence level (denoted X%). The confidence level is often specified before a test is conducted as part of a sample size calculation. We view the confidence level as equaling one minus the Type I error rate (α). A Type I error is committed when the null hypothesis is incorrectly rejected. An α value of 0.05 is typically used, corresponding to 95% confidence levels.
The p-value is used to determine if enough evidence exists to reject the null hypothesis in favor of the alternative. The p-value is the probability of incorrectly rejecting the null hypothesis.
The two possible conclusions, after assessing the data, are to: 1....