The Insignificance of Statistical Significance Testing
By Douglas H. Johnson1
Abstract: Despite their wide use in scientific journals such as The Journal of Wildlife Management, statistical hypothesis tests add very little value to the products of research. Indeed, they frequently confuse the interpretation of data. This paper describes how statistical hypothesis tests are often viewed, and then contrasts that interpretation with the correct one. I discuss the arbitrariness of P-values, conclusions that the null hypothesis is true, power analysis, and distinctions between statistical and biological significance. Statistical hypothesis testing, in which the null hypothesis about the properties of a population is almost always known a priori to be false, is contrasted with scientific hypothesis testing, which examines a credible null hypothesis about phenomena in nature. More meaningful alternatives are briefly outlined, including estimation and confidence intervals for determining the importance of factors, decision theory for guiding actions in the face of uncertainty, and Bayesian approaches to hypothesis testing and other statistical practices.
Key words: Bayesian approaches, confidence interval, null hypothesis, P-value, power analysis, scientific hypothesis test, statistical hypothesis test.
This resource is based on the following source (Northern Prairie Publication 1057):
Johnson, Douglas H. 1999. The Insignificance of Statistical Significance
Testing. Journal of Wildlife Management 63(3):763-772.
This resource should be cited as:
Johnson, Douglas H. 1999. The Insignificance of Statistical Significance
Testing. Journal of Wildlife Management 63(3):763-772. Jamestown,
ND: Northern Prairie Wildlife Research Center Online.
http://www.npwrc.usgs.gov/resource/methods/statsig/index.htm
(Version 16SEP99).
Editor's Note: Doug Johnson received The Wildlife Society Award for Outstanding Publication in Wildlife Ecology and Management, in the Article Category for this paper. The award was conferred at the Society's annual meeting, 13 September 2000, in Nashville, Tennessee.
| President of The Wildlife Society, Nova Silvy (right), presents Doug Johnson (left) with the Outstanding Publication Award. |
Table of Contents
-
Introduction
- What are More Extreme Data?
- Are Null Hypotheses Really True?
- P is Arbitrary
- Proving the Null Hypothesis
- Power Analysis
- Biological Versus Statistical Significance
- Other Comments on Hypothesis Tests
What is Statistical Hypothesis Testing?
Replication
What are the Alternatives
Conclusions
Acknowledgements
Literature Cited
1 U.S. Geological Survey, Biological Resources Division, Northern Prairie Wildlife Research Center, Jamestown, ND 58401, USA
E-mail: douglas_h_johnson@usgs.gov
Downloading Instructions -- Instructions on downloading and extracting files from this site.
statsig.zip
(40484 bytes) -- The Insignificance of Statistical Significance
Testing
Installation: Extract all files and open index.htm in a web browser.
