Good Transformations

Ambiguity and the NSF's Experiment with 'Transformative' Research

Just for the fun of it, imagine the following lines, sung to the tune of the Beach Boys’ “Good Vibrations” (if you need help with that, click here):

 Science is giving us expectations
We want game-changin’ permutations
Science is giving us expectations
We want radical transformations
Good, good, good transformations …

You’re now in the frame of mind of many within the science policy community—the state of mind that sees the transformative potential of science and technology as having little or no downside.

Increasingly within our scientific institutions, mere incremental advance is no longer sufficient. Scientists are being asked to be more radical, more willing to take risks, and more disposed to push the frontiers of knowledge—to become, in a word, transformative.

The U.S. National Science Foundation, or NSF, is one of several agencies developing initiatives to promote so-called “transformative” research. For instance, in September 2007 NSF issued Important Notice No. 130, which announced a change to NSF’s Intellectual Merit review criterion effective January 5, 2008. Reviewers would now be asked, “To what extent does the proposed activity suggest and explore creative, original, or potentially transformative concepts?” (bold indicates addition of “potentially transformative” to the criterion).

This announcement followed the release of a document (often called “The Butterfly Report” from the image on its cover) in May 2007 that contained the following definition of the term:

Transformative research involves ideas, discoveries, or tools that radically change our understanding of an important existing scientific or engineering concept or educational practice or leads to the creation of a new paradigm or field of science, engineering, or education. Such research challenges current understanding or provides pathways to new frontiers.

Good, good, good transformations

This definition was not meant to define strict boundaries between “transformative research,” or TR, and all the rest.[i] In a January 4, 2007, talk, “Transformative Research: The Artistry and Alchemy of the 21st Century,” then-Director of NSF Ardent Bement, Jr. suggested it would often be easy to identify transformative research in hindsight, but that we would continue to discuss the meaning of the term “transformative”:

But we also can – and no doubt will – continue to quibble among ourselves about the meaning of “transformative research,” which as yet has no universally accepted definition. That is just as it should be. When concepts as complex as “transformative research” are still emerging, we need to practice a kind of “constructive ambiguity.” Doing so will give us the flexibility to incorporate new knowledge and fresh perspectives as they arise; in other words, leave room for discovery. In that way, we can make course corrections along the way, adapt to changing circumstances, and remain open to diverse suggestions about the issues.

In March 2012 the three of us led a two-day workshop at NSF Headquarters to once again raise the question of the meaning of “transformative research.” TR has come to encapsulate an increasingly central question across both U.S. and foreign science agencies: In a hypercompetitive global economy, with pressing challenges in many areas (energy, food, water, disease, etc.), how can we do a better job of picking research projects that are true game changers?

The workshop

Workshop participants included 25 career scientists, engineers, historians, philosophers, and science policy researchers. Former and current NSF officials also attended parts of the workshop. One of the first questions discussed was, “What is meant by the term?” Bement’s insight concerning the advantages of there being no universally accepted meaning only holds if NSF’s definition is not accepted as comprehensive. Accordingly, the workshop spent little time on establishing the “right” definition of the term. Instead, we focused on fleshing out alternative interpretations that would provide both NSF and NSF researchers with tools to harness the “constructive ambiguity” of TR and turn it to their advantage.

The workshop did, however, focus on whether transformative research truly existed except retrospectively. Is TR merely a useful moniker for Congress and the public to indicate extra effort, or is there a real phenomenon here that might be identified on the front end?

Another central concern was, “Why transform?” The argument in favor of TR is that it will help us deal with our many urgent problems, while assisting the U.S. economy in the global race for economic advantage. On the other hand, many people today are uncomfortable with change, feeling with some justification that the changes in economics and culture over the last 30 years have left them with the short end of the stick.

Central to all these discussions was the work of Thomas Kuhn. In The Structure of Scientific Revolutions, Kuhn argued that revolutions in science are spontaneous, radical breaks with the status quo that ultimately transform a field of study. But Kuhn assumed that most science consisted of incremental puzzle solving, which he called “normal” science. What happens if science is constantly revolutionary? Again, some suggested that transformative research was the result of incremental advances within the current structure of normal science. If so, then NSF’s emphasis on TR is redundant: Transformations naturally occur within scientific fields in the course of normal, or incremental, research.

NSF has committed to emphasizing the importance of funding basic research that contributes to societal benefits, such as quality of life, economic growth, and environmental health, through the Broader Impacts merit review criterion. Thus the workshop also asked, “What is the relationship between TR and broader impacts?” Peer review is arguably the proper tool to advance a policy of promoting transformative research, but participants also noted that review panels could present a significant barrier to funding transformative science.

If what NSF wants is to fund transformative research, then why use reviewers who are committed to the current state of the art to try to find it? Recalling some of the issues NSF has had with incorporating broader impacts considerations into its merit-review process, when it comes to reviewing potentially transformative research, it is natural to ask, “Who really counts as a peer?” Political considerations aside, given that so many of the world’s greatest scientific discoveries flew in the face of conventional wisdom, how can peer-review panelists tell when proposed research will be potentially transformative? Can we identify transformative characteristics up front? And if a researcher specifies that the proposed research might be transformative, can reviewers trust the label?

The conversations launched at the workshop are only beginning. A white paper is being authored now, and we have plans to create a volume of essays on the topic. This volume will not be targeted only at academic specialists, but also at a broader audience of those concerned with public funding of research. Overall, transformative research presents us all with another opportunity to make publically funded science more accountable.

J. Britt Holbrook is research assistant professor of philosophy and assistant director at the Center for the Study of Interdisciplinarity, or CSID, at the University of North Texas, or UNT. Robert Frodeman is professor of philosophy at UNT and director of CSID. Kelli Barr is a graduate student in the UNT department of philosophy and graduate research assistant at CSID.

The authors gratefully acknowledge the support for this research provided by the NSF under grant No. 1129067, while emphasizing that any opinions, conclusions, and recommendations expressed here are those of the authors and do not necessarily reflect the views of NSF or any of its employees.

Comments on this article

By clicking and submitting a comment I acknowledge the Science Progress Privacy Policy and agree to the Science Progress Terms of Use. I understand that my comments are also being governed by Facebook's Terms of Use and Privacy Policy.