Fixing the Marcott Mess in Climate Science

In 1991 the National Research Council proposed what has come to be a widely accepted definition of misconduct in science:

Misconduct in science is defined as fabrication, falsification, or plagiarism, in proposing, performing, or reporting research. Misconduct in science does not include errors of judgment; errors in the recording, selection, or analysis of data; differences in opinions involving the interpretation of data; or misconduct unrelated to the research process.

Arguments over data and methods are the lifeblood of science, and are not instances of misconduct.

However, here I document the gross misrepresentation of the findings of a recent scientific paper via press release which appears to skirt awfully close to crossing the line into research misconduct, as defined by the NRC. I recommend steps to fix this mess, saving face for all involved, and a chance for this small part of the climate community to take a step back toward unambiguous scientific integrity.

The paper I refer to is by Marcott et al. 2013, published recently in Science. A press release issued by the National Science Foundation, which funded the research, explains the core methodology and key conclusion of the paper as follows (emphasis added):

Peter Clark, an OSU paleoclimatologist and co-author of the Science paper, says that many previous temperature reconstructions were regional and not placed in a global context.

"When you just look at one part of the world, temperature history can be affected by regional climate processes like El Niño or monsoon variations," says Clark.

"But when you combine data from sites around the world, you can average out those regional anomalies and get a clear sense of the Earth's global temperature history."

What that history shows, the researchers say, is that during the last 5,000 years, the Earth on average cooled about 1.3 degrees Fahrenheit--until the last 100 years, when it warmed about 1.3 degrees F.

The press release clearly explains that the paper (a) combines data from many sites around the world to create a "temperature reconstruction" which gives a "sense of the Earth's temperature history," and (b) "that history shows" a cooling over the past 5000 years, until the last 100 years when all of that cooling was reversed.

The conclusions of the press release were faithfully reported by a wide range of media outlets, and below I survey several of them to illustrate that the content of the press release was accurately reflected in media coverage and, at times, amplified by scientists both involved and not involved with the study.

Examples of Media Coverage

Here is Justin Gillis at the New York Times, with emphasis added to this excerpt and also those further below:

The modern rise that has recreated the temperatures of 5,000 years ago is occurring at an exceedingly rapid clip on a geological time scale, appearing in graphs in the new paper as a sharp vertical spike.

Similarly, at the NY Times Andy Revkin reported much the same in a post titled, "Scientists Find an Abrupt Warm Jog After a Very Long Cooling." Revkin included the following graph from the paper along with a caption explaining what the graph shows:

Revkin's caption:  A new Science paper includes this graph of data providing clues to past global temperature. It shows the warming as the last ice age ended (left), a period when temperatures were warmer than today, a cooling starting 5,000 years ago and an abrupt warming in the last 100 years.

Revkin concluded: "the work reveals a fresh, and very long, climate “hockey stick.”" For those unfamiliar, a hockey stick has a shaft and a blade.

Any association with the so-called "hockey stick" is sure to capture interest in the highly politicized context of the climate debate, in which the iconic figure is like catnip to partisans on both sides. Here is Michael Lemonick at Climate Central:

The study... confirms the now famous “hockey stick” graph that Michael Mann published more than a decade ago. That study showed a sharp upward temperature trend over the past century after more than a thousand years of relatively flat temperatures. . .

“What’s striking,” said lead author Shaun Marcott of Oregon State University in an interview, “is that the records we use are completely independent, and produce the same result.”

Here is Grist.org, which refers in the passage below to the same figure shown above:

A study published in Science reconstructs global temperatures further back than ever before — a full 11,300 years. The new analysis finds that the only problem with Mann’s hockey stick was that its handle was about 9,000 years too short. The rate of warming over the last 100 years hasn’t been seen for as far back as the advent of agriculture.

 To be clear, the study finds that temperatures in about a fifth of this historical period were higher than they are today. But the key, said lead author Shaun Marcott of Oregon State University, is that temperatures are shooting through the roof faster than we’ve ever seen.

“What we found is that temperatures increased in the last 100 years as much as they had cooled in the last 6,000 or 7,000,” he said. “In other words, the rate of change is much greater than anything we’ve seen in the whole Holocene,” referring to the current geologic time period, which began around 11,500 years ago.

Back to more mainstream outlets, here is how Nature characterized the study, offering a substantially similar but somewhat more technical description of the curve shown in the figure above:

Marcott and his colleagues set about reconstructing global climate trends all the way back to 11,300 years ago, when the Northern Hemisphere was emerging from the most recent ice age. To do so, they collected and analysed data gathered by other teams. The 73 overlapping climate records that they considered included sediment cores drilled from lake bottoms and sea floors around the world, along with a handful of ice cores collected in Antarctica and Greenland.

Each of these chronicles spanned at least 6,500 years, and each included a millennium-long baseline period beginning in the middle of the post-ice-age period at 3550 bc.

For some records, the researchers inferred past temperatures from the ratio of magnesium and calcium ions in the shells of microscopic creatures that had died and dropped to the ocean floor; for others, they measured the lengths of long-chain organic molecules called alkenones that were trapped in the sediments.

After the ice age, they found, global average temperatures rose until they reached a plateau between 7550 and 3550 bc. Then a long-term cooling trend set in, reaching its lowest temperature extreme between ad 1450 and 1850. Since then, temperatures have been increasing at a dramatic clip: from the first decade of the twentieth century to now, global average temperatures rose from near their coldest point since the ice age to nearly their warmest, Marcott and his team report today in Science.

And here is New Scientist, making reference to the exact same graph:

Shaun Marcott of Oregon State University in Corvallis and colleagues have compiled 73 such proxies from around the world, all of which reach back to the end of the last glacial period, 11,300 years ago. During this period, known as the Holocene, the climate has been relatively warm – and civilisation has flourished.

"Most global temperature reconstructions have only spanned the past 2000 years," says Marcott.

Marcott's graph shows temperatures rising slowly after the ice age, until they peaked 9500 years ago. The total rise over that period was about 0.6 °C. They then held steady until around 5500 years ago, when they began slowly falling again until around 1850. The drop was 0.7 °C, roughly reversing the previous rise.

Then, in the late 19th century, the graph shows temperatures shooting up, driven by humanity's greenhouse gas emissions.

The rate of warming in the last 150 years is unlike anything that happened in at least 11,000 years, says Michael Mann of the Pennsylvania State University in University Park, who was not involved in Marcott's study. It was Mann who created the original hockey stick graph (see upper graph here), which showed the change in global temperatures over the last 1000 years.

Over the Holocene, temperatures rose and fell less than 1 °C, and they did so over thousands of years, says Marcott. "It took 8000 years to go from warm to cold." Agriculture, communal life and forms of government all arose during this relatively stable period, he adds. Then in 100 years, global temperatures suddenly shot up again to very close to the previous maximum.

It seems clear that even as various media took different angles on the story and covered it in varying degrees of technical detail, the articles listed above accurately reflected the conclusions reflected in the NSF press release, and specifically the "hockey stick"-like character of the new temperature reconstruction. Unfortunately, all of this is just wrong, as I explain below. (If you'd like to explore media coverage further here is a link to more stories. My colleague Tom Yulsman got punked too.)

The Problem with the NSF Press Release and the Subsequent Reporting

There is a big problem with the media reporting of the new paper. It contains a fundamental error which (apparently) originates in the NSF press release and which was furthered by public comments by scientists.

In a belatedly-posted FAQ to the paper, which appeared on Real Climate earlier today, Marcott et al. make this startling admission:

Q: What do paleotemperature reconstructions show about the temperature of the last 100 years?

A: Our global paleotemperature reconstruction includes a so-called “uptick” in temperatures during the 20^th-century. However, in the paper we make the point that this particular feature is of shorter duration than the inherent smoothing in our statistical averaging procedure, and that it is based on only a few available paleo-reconstructions of the type we used. Thus, the 20^th century portion of our paleotemperature stack is not statistically robust, cannot be considered representative of global temperature changes, and therefore is not the basis of any of our conclusions.

Got that?

In case you missed it, I repeat:

. . . the 20^th century portion of our paleotemperature stack is not statistically robust, cannot be considered representative of global temperature changes . . .

What that means is that this paper actually has nothing to do with a "hockey stick" as it does not have the ability to reproduce 20th century temperatures in a manner that is "statistically robust." The new "hockey stick" is no such thing as Marcott et al. has no blade. (To be absolutely clear, I am not making a point about temperatures of the 20th century, but what can be concluded from the paper about temperatures of the 20th century.)

Yet, you might recall that the NSF press release said something quite different:

What that [temperature reconstruction] history shows, the researchers say, is that during the last 5,000 years, the Earth on average cooled about 1.3 degrees Fahrenheit--until the last 100 years, when it warmed about 1.3 degrees F.

So what the paper actually shows is the following, after I have removed from the graph the 20th century period that is "not statistically robust" (this is also the figure that appears at the top of this post):

Surely there is great value in such an analysis of pre-20th century temperatures. And there can be no doubt there will be continuing debates and discussions about the paper's methods and conclusions. But one point that any observer should be able to clearly conclude is that the public representation of the paper was grossly in error. The temperature reconstruction does not allow any conclusions to be made about the period after 1900.

Does the public misrepresentation amount to scientific misconduct? I'm not sure, but it is far too close to that line for comfort. Saying so typically leads to a torrent of angry ad hominem and defensive attacks, and evokes little in the way of actual concern for the integrity of this highly politicized area of science. Looking past the predictable responses, this mess can be fixed in a relatively straightforward manner with everyone's reputation intact.

How to Fix This

Here are the steps that I recommend should be taken:

1) Science should issue a correction to the paper, and specially do the following:

(a) retract and replot all figures in the paper and SI eliminating from the graphs all data/results that fail to meet the paper's criteria for "statistical robustness."
(b) include in the correction the explicit and unambiguous statement offered in the FAQ released today that the analysis is not "statistically robust" post-1900.

2) NSF should issue a correction to its press release, clarifying and correcting the statements of Peter Clark (a co-author, found above) and Candace Major, NSF program manager, who says in the release:

"The last century stands out as the anomaly in this record of global temperature since the end of the last ice age," says Candace Major, program director in the National Science Foundation's (NSF) Division of Ocean Sciences.

3) The New York Times (Gillis and Revkin, in particular), Nature and New Scientist as outlets that pride themselves in accurate reporting of science should update their stories with corrections. Grist and Climate Central should consider the same.

[UPDATE: Andy Revkin at DotEarth has updated his posts here and here to reference the "lost blade" from the hockey stick and link to this post. That was quick and easy. Others take note.]

Let me be perfectly clear -- I am accusing no one of scientific misconduct. The errors documented here could have been the product of group dynamics, institutional dysfunction, miscommunication, sloppiness or laziness (do note that misconduct can result absent explicit intent). However, what matters most now is how the relevant parties respond to the identification of a clear misrepresentation of a scientific paper by those who should not make such errors.

That response will say a lot about how this small but visible part of the climate community views the importance of scientific integrity.