Hypothesis Testing and Proof by

Hypothesis Testing and Proof by Contradiction: An Analogy C A. REEVES and J. K. BREWER

Concerned statistics instructors are constantly on the lookout for examples and analogies to assist students of basic statistics to comprehend the underpinnings of hypothesis testing. Logic and mathematics are the foundations of statistical theory and as such pervade the entire structure of statistics providing several useful comparisons. One such analogy, the "proof by contradiction" in mathematics, provides a reasonable parallel to the concept of hypothesis testing and in the process helps students see the fallacy of believing that the null hypothesis, H₀, is true when they have simply failed to reject it.

Recall that a mathematical proof by contradiction can be briefly described as:

I. Assume the truth of a statement (perhaps A = B, or A <B).

2. Using only accepted rules of logic, legitimate mathematical manipulations, and reasonable assumptions, derive an expression which contradicts either a known fact (2 = I) or the original statement (A not equal B, or A >B).

3. Conclude that the original assumption must in fact be untrue since it leads to an untenable position. Accept, then, the denial of the original statement as fact. QED.

Analogously, the essence of hypothesis testing can be described as:

I. Assume the truth of a null hypothesis, H₀.

2. Collect data (the amount of data depending on error rates a and b, as well as effect size), make assumptions appropriate to the statistical test to be utilized, and then calculate the probability of the data (or some summarizing function of the data) given that H₀ is true.

3. If the probability above (denoted by p in most statistical literature) is "quite small," then reject H₀. Note that "quite small" is usually taken to be any value less than or equal to a.

It is obvious that the hypothesis testing procedure does not constitute a "proof’ in the mathematical sense, as do the steps in the contradiction argument; however, the analogy is fairly complete. In any "proof by contradiction" the truth of the assumption is brought into question by resulting statements which are mathematically contrary to the original assumption or an accepted fact; consequently, the initial assumption is rejected. In hypothesis testing, the truth of H₀ is doubted if it is unlikely for the factual data to have occurred with H₀ being true. Therefore the statement H₀ is rejected. Stated another way, in contradiction proofs, a statement is rejected when facts resulting therefrom produce an undeniable contradiction; in hypothesis testing, a statement is rejected when data-based probabilities derived from this statement cast serious doubt on the truth of the statement.

It should be relatively clear, then, why a failure to reject H₀ does not constitute a verification of its truth. It simply means that the researcher has failed to provide evidence sufficient to cast serious doubt on the truth of H₀. Analogously, one does not assume A = B simply because one cannot produce contradictory evidence. A mathematician or statistician would be taken less than seriously if he or she concluded that A= B because he or she could not show otherwise. The reasoning is no less logical in hypothesis testing in that a failure to reject H₀should not lead the researcher to behave as if H₀had been verified.

Semantics begins to play a role at this point, for some writers use "accept H₀" when a failure to reject H₀ occurs. As long as "accept H₀" does not connote a verification of H₀ or convey a belief that H₀ is true, this usage is permissible. If not, the longer but more accurate statement "failed to reject H₀" should be used.

Back to contents of The Best of Teaching Statistics
Back to main Teaching Statistics page