By Dr. Mike Clark, Consultant

September 2007 – Many people who ask for our help do so in part, whether consciously or not, because they are not exactly aware of what statistics is. As such I have cobbled this list together in hopes of making that at least a little clearer. The list is by no means definitive nor exhaustive, but does hit on what I perceive to be some basic misunderstandings prevalent in both the applied setting and to the uninitiated. By statistics I mean all parts of a statistical analysis from conceptualization through interpretation and as the chosen tool of scientific investigation. Clicking on the numbers will take you to the corresponding clarification.

Some things the field of statistics is not.

Math. “Statistics is a science in my opinion, and it is no more a branch of mathematics than are physics, chemistry and economics; for if its methods fail the test of experience—not the test of logic—they are discarded." From "The Growth of Experimental Design in a Research Laboratory", 1953.

A Collection of Heuristics. While rules of thumb have their place, engaging in statistical analysis is not simply deeming ‘meaningful’ to anything with X value (whatever the metric), doing X analysis whenever the variables are just so, etc.  In some sciences this is only a recent revelation to those on the applied side, and others still have yet to get to that understanding.  No research endeavor is an island, and any analysis must be thoughtfully interpreted in light of relevant history, current context and future implications.

Getting Output:  Getting statistical output has been rendered so easy by technology it could be produced by children with some programs.  Just because one has some output doesn’t mean one has done a statistical analysis.  If this is confusing to you as to why you need more training.

Done quickly or something for which there is a ‘quick’ answer. If you’re doing a proper statistical analysis, it will never be something you can do quickly if you want to do it well. See this related Research Matters article by Dr. Richard Herrington. Also, people ask us ‘quick questions’ all the time.  We’ve yet to ever find a quick answer that would be adequate.

Easy.  This goes with #4 to some extent.  Even the simplest of techniques (e.g. calculating a ‘middle’) has several possibilities available (median, mode, mean, winsorized mean, M-estimators etc.), and some might be equally viable given a situation.   A simple independent samples t-test could be done standard, robustly (e.g. trimmed means), nonparametrically (e.g. bootstrapped), determined via a confidence interval for the mean difference, approached purely from an effect size perspective etc.  If you think something is able to be done quickly and easily, odds are you have taken a severely limited approach.

Impossible to Learn.  Learning statistical concepts can certainly be difficult, but it certainly is not impossible for anyone to learn.  It will take time though, and if one equates time with overwhelming difficulty, and many do, then perhaps it is impossible for that person, as much of life probably is.

Disconnected from Theory.  I’ve often heard people claim to be good with theory and bad with statistics.  I have yet to understand how that is possible.  No scientific theory gets by without an implication of measurement, and if a theory is devised without an understanding of how it would be measured, and thus eventually analyzed, it is not a scientific one.  As such from the very get go methodological and analytical considerations should be at the forefront of any research project, and be continuously considered from that beginning to the end. One of the more common and biggest problems we come across is data that has no connection to the theory a client is interested in.

Expensive.  It can be, but it doesn’t have to be.  There are alternatives, and one such alternative is R, which is free and which we support and use quite extensively.  For much of our needs it does everything and more than the expensive stuff.  If you don’t like the price for something there are options.  Here are some.

Some things Statistics Is

An essential part of scientific inquiry. There is no science without measurement, and no understanding of the measurement without statistics applied by a keen observer.  It is the analysis that enables us to extract the knowledge hiding within the data.

Something that entails critical thinking. This part really gets folks that tend to think statistical analysis is many of those things listed in the ‘not’ column.  If you don’t like to think hard, you shouldn’t be doing statistical analyses or trying to interpret them.

In a constant state of development. People seem to think that their introduction to statistics at the undergrad or graduate level is some sort of overview of the field.  Something to take note of: anything in a typical stats textbook involves methods that at the very least are probably a quarter century old, contains many and perhaps most that are well older than that, and omits the vast majority of the field.  Things have changed since Fisher’s days—a lot.

A source for new ideas. The result of a good analysis is always a springboard for new ways to think about things and a fuller way of understanding previous findings from some research domain.  If it does not lead to new ways of thinking you can assume something has gone wrong.  As a hint: non-‘significant’ results are still meaningful.

Exploratory as well as confirmatory. Description and exploration are part and parcel of science.  One can gain a tremendous amount of insight from an initial glimpse into a research domain as well simply examining the data to the fullest.  For some reason that fell out of favor in some areas of science and/or was not deemed to be ‘enough’ (for theses, dissertations, publications), or somehow cheating if you explore your own data.  A good exploration can be as good as any confirmatory one, and oftentimes can be more useful.

A means by which to understand causal structure. If one thinks causal explanations are not possible in a domain of scientific research, they have a gross misunderstanding of both science and causality.  There is a continuum of confidence in causal statements, but the goal of science and its methods is to assign causal attributions to natural events.

A means by which to reduce uncertainty. There are lots of things we don’t know.  Science and its methods are the most viable means with which to reduce our ignorance regarding ourselves and the world around us.

Mathematically intensive. Statistics uses a lot of math. So does everyday life, what’s the big deal?

Interpretive. Whatever the results are, different interpretations are available and perhaps equally plausible based on the experiences and knowledge of interested parties.  This is a good thing, not a reason to throw our hands up in frustration.

A very useful tool. In the end, statistical analysis is simply the tool we use to extract meaning from a great deal of information we can not possibly comprehend otherwise.  Seen as such it should hardly be seen as something aversive, nor should there be an inherent negativity or suspicion associated with it.  Save that for politics.

All of this may sound like just employment-related bias, though I might mention that I would not refer to myself as a statistician, and much of my experience comes from a social science realm where perhaps this clarification is more needed.  However, it doesn’t take much to see there are plenty out there in various domains of research that are content to play a game.  Some, perhaps many, perhaps a majority of people in some areas of research take a minimum effort approach to their methods and analysis, then try to convince others about what they have supposedly discovered and concluded from those lackadaisical efforts.  While there is enough good research being done to sustain progress, it is slowed considerably by such shenanigans.  Furthermore, those reading others' works who have the aforementioned misunderstandings of statistics restrict themselves to having to believe whatever the authors conclude or simply distrusting whatever is presented, not a very good state of affairs, but unfortunately one the general public has to deal with.

The causes and possible solutions to these problems I will have to leave for another article.  For now, it is hoped that the clarification will provide some insight for those that we assist and others who might come across it.  Be seeing you.