*UPDATE IV* Eight Worst Inputs Used in Colorado Health Study

UPDATE IV (5/16/12, 9:12am ET): The Colorado School of Public Health’s paper on hypothetical future health impacts of natural gas development scored an nine-minute profile yesterday on NPR. That’s practically an eternity in broadcast journalism, more than enough time for a detailed discussion on both the CSPH’s conclusions and the many criticisms of their work.

Unfortunately, NPR failed its audience. Reporter Elizabeth Shogren accepted just about everything the CSPH said at face value, and only briefly mentions that its findings have been challenged by local officials and scientists within the industry. The segment conveniently fails to mention why those challenges were made, and what the substance of those challenges were – effectively denying NPR’s listeners a full account of the controversy.

Other news organizations managed to describe the findings and some of the flaws in the CSPH’s paper, in accordance with basic journalistic standards. Among the examples:

The Associated Press – “Energy In Depth has disputed that study’s findings, saying it exaggerates emissions from gas well development by at least 10 times and fails to take into account exhaust fumes from a nearby interstate highway.”

Public Radio International – “[A]n oil and natural gas industry group has mounted an aggressive campaign to point out what it says are flaws in the research. … Among the group’s specific complaints are that the study doesn’t account for pollution generated by Interstate 70, which passes within a mile of the wells.”

Denver Business Journal – “Among the study’s problems, according to Energy-In-Depth and [the Colorado Oil & Gas Association] … air samples for the study were taken between January 2008 to November 2010, but Colorado tightened its air emission requirements for the oil and gas industry in April 2009 … Air pollution from other sources wasn’t considered as a source, even though some of the air samples were taken one mile away from Interstate 70 … The cancer risks outlined in the report are no greater than national statistical averages…”

UPI – “The study used out-of-date emissions data and overestimated by a factor of 10 how long it takes to develop a new natural gas well, the industry group said, adding that the study failed to account for pollution from Interstate 70, a mile away from the gas wells.”

So then, what are the main issues with the CSPH paper? Here’s the short version: CSPH dramatically overstates people’s exposure to gas-well emissions. But even though they assume emissions are more than 10 times higher than real-world conditions, and treat exhaust fumes from Interstate 70 as if they came from gas wells, the authors of the paper admit they could not show a clear correlation between proximity to those gas wells and higher health risks. By deciding the public didn’t need to hear those criticisms, NPR denied Colorado’s oil and gas industry the chance to properly defend itself against the allegations in this flawed research paper.

Of course, beyond the industry, lots of other folks have stepped up and identified serious flaws with the CSPH methodology. Let’s start with Jim Rada, Garfield County’s top public health official. Rada was interviewed for NPR’s segment on the CSPH, but there’s no mention that he disavowed its paper on health risks. After the paper was released, Rada made clear that it wasn’t sanctioned or funded by Garfield County.

Then there’s the Colorado Department of Public Health and Environment. At one point, the CDPHE was helping the CSPH find funding for its work, but the agency eventually backed out after criticizing CSPH’s approach. NPR surely knows this, because it interviewed David Ludlam, the executive director of the West Slope Colorado Oil & Gas Association. WSCOGA’s March 20 rebuttal to the CSPH paper specifically cited the CDPHE’s “extremely critical” comments about the project’s methodology.

So, for the record, the health-risk study that underpins all nine minutes of this NPR segment has been criticized by multiple stakeholders, not just the oil and gas industry. The study’s critics also include local and state health officials, which would seem a relevant thing to mention in a nine-minute NPR segment on the study. But it never was. Probably because it didn’t fit with the reporter’s pre-established narrative.

UPDATE III (5:01pm ET, 3/23/2012): In a story for Platts Gas Daily (subs. req’d), Garfield County environmental health chief Jim Rada says the press release announcing the CSPH study “made it sound like there was new information” that was being released. In reality, the data in the study was simply lifted from an earlier draft assessment that was criticized by the state and the industry alike. “One needs to read the entire report to understand the relative risks,” Rada says. “There are significant limitations to the data.”

UPDATE II (1:22pm ET, 3/22/2012): The Colorado Oil and Gas Association (COGA) issued a statement that casts more light onto the flawed assumptions in the CSPH study. COGA states up front that it “appreciates the work being done by Colorado’s institutions of higher education and relies on them for unbiased, non-politicized data and information.” Unfortunately, the CSPH report “does not reach that standard.” COGA adds that the initial study on this subject by CSPH “was critiqued as politically-charged and of questionable scientific merit,” and, unfortunately, “the data from that [initial] report is regurgitated in the current study.”

UPDATE (2:35pm ET, 3/21/2012): David Ludlam, executive director of the West Slope Colorado Oil and Gas Association (WSCOGA), posted a response to the CSPH findings. “WSCOGA agrees that oil and gas operations must be protective of public health and the environment,” Ludlam writes. But Ludlam also notes that this particular report “lacks the necessary supportive data and proper context and represents a basic restatement of the Battlement Mesa Health Impact Assessment — a project that was suspended, in part, due to the State of Colorado’s criticism of the project.” Ludlam adds that the report is “basically a restatement of data already proven to be weak on supportive data and void of critical proper context.”

—Original post from March 20, 2012—

A new paper from the Colorado School of Public Health (CSPH) released this week suggests the development of oil and natural gas in general – and the use of hydraulic fracturing in particular – can cause “serious health impacts” for those who live closest to well sites. But if you look past the ominous headlines that the study launch generated and examine the range of strange assumptions that form the basis for the report, the conclusions are not only rendered fairly predictable, but also unquestionably flawed.

Of course, we’ve all seen first-hand how choices made by a researcher with respect to the inputs he or she uses as part of a study plan can, and indeed will, significantly impact the nature of the results. The infamous Howarth paper from Cornell University, for example, used a global warming potential for methane 45 percent greater than what even the U.N. Intergovernmental Panel on Climate Change says is appropriate. That flawed input, along with the casual use of “inflated leakage” rates, served as the central basis for its deeply flawed (and widely debunked) conclusions.

In some ways, the inputs used by the CSPH researchers are even more egregious than those found in the Cornell report. Below, we identify eight specific assumptions made that, upon closer examination and considered in combination, cast serious doubt on the results produced by the modeling exercise.

Bad Input #1: Out of Date Emissions Data

CSPH: “We used air toxics data collected in Garfield County from January 2008 to November 2010 as part of a special study of short-term exposure as well as on-going ambient air monitoring program data to estimate subchronic and chronic exposures and health risks.” (p. 8 )

• FACT: Colorado updated its regulatory requirements for oil and gas systems in February 2009, which means at least a portion of the data collected by CSPH is from an operating environment that, by law, no longer exists. Among the rules were requirements for volatile organic compounds (VOCs) to be reduced by as much as 95 percent through the use of low- or no-bleed pneumatic devices.

Bad Input #2: Inflated Time to Drill and Complete a Well by as Much as 900%

CSPH: “We assumed a 30-year project duration based on an estimated 5-year well development period for all well pads, followed by 20 to 30 years of production.” (p. 11)

CSPH: “To evaluate subchronic non-cancer HIs from well completion emissions, we estimated that a resident lives ≤ 1/2 mile from two well pads resulting a 20- month exposure duration based on 2 weeks per well for completion and 20 wells per pad, assuming some overlap between activities.” (p. 12)

• FACT: The well development process takes a matter of months, not years. In fact, well development, as defined by CSPH in the same study, “involves pad preparation, well drilling, and well completion” (p. 5). According to the Marcellus Center at Penn State University: “The total time to drill each well is about a three to six weeks depending on the depth and length of the horizontal well, so if there are four wells on a well pad, you could expect the big rig to be there for about three to six months.” The Marcellus Center adds that hydraulic fracturing (i.e. completion) “typically occurs within a few weeks or months of the well drilling, dependent on the project schedule, and may take up to several days for each well to be hydraulically fractured.” API also notes that the process takes “two to five days for the entire multi-stage fracturing operation.”

Bad Input #3: Inflated Small Cancer Risks Due to Lack of Context

CSPH: “The cumulative cancer risks based on the 95% UCL of the mean concentration were 6 in a million for residents > ½ mile from wells and 10 in a million for residents < ½ mile from wells.” (p. 15-16)

• FACT: While these numbers are small, the lack of context suggests they could be significant. But according to EPA’s National-Scale Air Toxics Assessment (NATA), these risks are in line with or even well below the risk for the entire U.S. population. According to a recent EPA report: “NATA estimates that all 285 million people in the U.S. have an increased cancer risk of greater than 10 in one million. 13.8 million people (less than 5 percent of the total U.S. population based on the 2000 census) have an increased cancer risk of greater than 100 in a million. The average, national, cancer risk for 2005 is 50 in a million. This means that, on average, approximately 1 in every 20,000 people have an increased likelihood of contracting cancer as a result of breathing air toxics from outdoor sources if they were exposed to 2005 emission levels over the course of their lifetime.”

Bad Input #4: Assumed No One Ever Leaves Garfield County

CSPH: “We assumed a resident lives, works, and otherwise remains within the town 24 hours/day, 350 days/year and that lifetime of a resident is 70 years, based on standard EPA reasonable maximum exposure (RME) defaults (US EPA 1989)” (p. 11)

• FACT: The study assumes that, aside from a few quick out-of-town weekend trips per year, residents never, ever – ever! – leave the city limits over the course of 70 years. Unless the “town” is actually a prison, this is a fundamentally flawed assumption about the length and extent of exposure.

Bad Input #5: Failed to Account/Control for Other Variables

CSPH: “The GCPH collected ambient air samples every six days between January 2008 and November 2010 (163 samples) from a fixed monitoring station located in the midst of rural home sites and ranches and NGD, during both the well development and production. The site is located on top of a small hill and 4 miles upwind of other potential emission sources, such as a major highway (Interstate-70) and the town of Silt, CO…” (p. 9)

CSPH: “The GCPH collected 16 ambient air samples at each cardinal direction along 4well pad perimeters (130 to 500 feet from the well pad center) in rural Garfield County during well completion activities… All five well pads are located in areas with active gas production, approximately one mile from Interstate-70.” (p. 9-10)

• FACT: When studying concentrations and identifying sources of benzene, it’s probably not a great idea to take samples from areas closer to a major highway than the ambient, control samples. The EPA classifies benzene as one of many Mobile Source Air Toxics (MSATs), and in its Final Rule to Reduce Mobile Source Air Toxics, the EPA notes that “most of the nation’s benzene emissions come from mobile sources. People who live or work near major roads, or spend a large amount of time in vehicles, are likely to have higher exposures and higher risks. People living in homes with attached garages are also likely to be exposed to benzene levels that are higher than average.”

Bad Input #6: Poor Distance Assumptions Increased Uncertainty

CSPH: “The actual distance at which residents may experience greater exposures from air emissions may be less than or greater than a 1/2 mile, depending on dispersion and local topography and meteorology. This lack of spatially and temporally appropriate data increases the uncertainty associated with the results.” (p. 21)

• FACT: Here, CSPH admits that its main basis of comparison – those living within and outside of a half mile zone surrounding a well – may not actually be representative after all. And in a study whose main conclusion, according to its press release, is “air emissions near fracking sites may have serious health impacts,” uncertainty about distance – and thus what defines “near” – means there is also considerable uncertainty about the conclusions.

Bad Input #7: Failed to Communicate with Local Environmental Officials

CSPH study author Lisa Mackenzie: Garfield County “did not financially support the scientific paper. We did this on our own. We feel the findings are significant, and we are scientists, and this is the way scientists communicate with each other” (Glenwood Springs Post Independent, 3/20/2012).

• FACT: If that’s how scientists communicate, it’s news to at least one notable health official, namely Jim Rada, Garfield County’s chief environmental health official. Rada said of Mackenzie’s work: “I had no knowledge of what she was studying, or her methods, or the implications of her work.” Rada also noted: “We are not in violation on ambient air quality standards.”

Bad Input #8: Who’s in Charge Here?

CSPH press release: “Garfield County asked the Colorado School of Public Health to assess the potential health impacts of these wells on the community of Battlement Mesa with a population of about 5,000.” (Press Release, 3/19/2012)

• FACT: This is also news to Rada, who said: “We didn’t ask them to do this paper. They were not sanctioned by the county, or paid by the county to do this paper.” As the Glenwood Springs Post Independent reported shortly after the release of the study, the CSPH paper “became embroiled in controversy” about a year ago after criticism by the Colorado Department of Public Health and Environment (among others), and the study “was decommissioned by the Garfield County commissioners in May 2011.”