Category Archives: Science Policy

Modernising Research Monitoring in Europe | Center for the Science of Science & Innovation Policy

Posted on by

The tracking of the use of research has become central to the measurement of research impact. While historically this tracking has meant using citations to published papers, the results are old, biased, and inaccessible – and stakeholders need current data to make funding decisions. We can do much better. Today’s users of research interact with that research online. This leaves an unprecedented data trail that can provide detailed data on the attention that specific research outputs, institutions, or domains receive.

However, while the promise of real time information is tantalizing, the collection of this data is outstripping our knowledge of how best to use it, our understanding of its utility across differing research domains and our ability to address the privacy and confidentiality issues. This is particularly true in the field of Humanities and Social Sciences, which have historically been under represented in the collection of scientific corpora of citations, and which are now under represented by the tools and analysis approaches being developed to track the use and attention received by STM research outputs.

We will convene a meeting that combines a discussion of the state of the art in one way in which research impact can be measured – Article Level and Altmetrics – with a critical analysis of current gaps and identification of ways to address them in the context of Humanities and Social Sciences.

Modernising Research Monitoring in Europe | Center for the Science of Science & Innovation Policy.

Reflections on the 2014 Carey Lecture at the AAAS Forum on S&T Policy

Posted on by

Cherry A. Murray delivered the Carey Lecture last night at this year’s AAAS Forum on S&T Policy. I want to address one aspect of her talk here — the question of transdisciplinarity (TD, which I will also use for the adjective ‘transdisciplinary’) and its necessity to address the ‘big’ questions facing us.

As far as I could tell, Murray was working with her own definitions of disciplinary (D), multidisciplinary (MD), interdisciplinary (ID), and TD. In brief, according to Murray, D refers to single-discipline approaches to a problem, ID refers to two disciplines working together on the same problem, MD refers to more than two disciplines focused on the same problem from their own disciplinary perspectives, and TD refers to more than two disciplines working together on the same problem. Murray also used the term cross-disciplinary, which she did not define (to my recollection).

All these definitions are cogent. But do we really need a different term for two disciplines working on a problem together (ID) and more than two disciplines working on a problem together (TD)? Wouldn’t it be simpler just to use ID for more than one discipline?

I grant that there is no universally agreed upon definition of these terms (D, MD, ID, and TD). But basically no one who writes about these issues uses the definitions Murray proposed. And there is something like a rough consensus on what these terms mean, despite the lack of universal agreement. I discuss this consensus, and what these definitions mean for the issue of communication (and, by extension, cooperation) between and among disciplines here:10.1007/s11229-012-0179-7. 

I tend to agree that TD is a better approach to solving complex problems. But in saying this, I mean more than involving more than two disciplines. I mean involving non-academic, and hence non-disciplinary, actors in the process. It’s actually closer to the sort of design thinking that Bob Schwartz discussed in the second Science + Art session yesterday afternoon.

One might ask whether this discussion of terms is a distraction from Murray’s main point — that we need to think about solutions to the ‘big problems’ we face. I concede the point. But that is all the more reason to get our terms right, or at least to co-construct a new language for talking about what sort of cooperation is needed. There is a literature out there on ID/TD, and Murray failed to engage it. To point out that failure is not to make a disciplinary criticism of Murray (as if there might be a discipline of ID/TD, a topic I discuss here). It is to suggest, however, that inventing new terms on one’s own is not conducive to the sort of communication necessary to tackle the ‘big’ questions.

Ahead of the Curve // John J. Reilly Center // University of Notre Dame

Posted on by

Ahead Of The Curve: Anticipating Ethical, Legal, and Societal Issues Posed by Emerging Weapons Technologies

April 22-23, 2014

University of Notre Dame

“Ahead of the Curve” will provide a forum to discuss the “action-oriented” chapters of the soon-to-be-released National Academy of Science’s report, “Emerging and Readily Available Technologies and National Security.” The report was commissioned by the Defense Advanced Research Projects Agency (DARPA) in order to begin a discussion about the conduct and applications of research on military technology as well as their unforseen and inadvertant consequences. Speakers will include members of the NAS committee that wrote the report, along with distinguished experts on the ethics, law, and social impacts of new weapons technologies and representatives of agencies and organizations that are home to cutting-edge weapons research. Presentations will address the ethical, legal, and societal issues that policy makers, researchers, and industries need to anticipate as new technologies arise, specifically in fields such as robotics, autonomous systems, prosthetics and human enhancement, cyber weapons, information warfare technologies, synthetic biology, and nanotechnology. Our primary goal is to help government agencies, institutions, and researchers grow the expertise necessary for early and continuing engagement with the ethical, legal, and societal implications of new weapons technologies as they are planned and developed. We also aim to generate a broad public audience for the NAS report, this being an area in which public education is necessary, as is elevating the level of factually well-informed, public discourse.

via Ahead of the Curve // John J. Reilly Center // University of Notre Dame.

Measuring the Impacts of Science | AAAS Forum on Science and Technology Policy

Posted on by

I’m looking forward to moderating a panel on day 1 of the AAAS Forum on Science and Technology Policy.

2:00 Current Issues in S&T Policy (Breakout Sessions) 
 
(A) Measuring the Impacts of Science   
• What are the policy relevant challenges, tools, and approaches to measuring the social impact of scientific research? • How can improved indicators capture change in science, technology, and innovation? • Are altmetrics the solution to measuring social impacts? 
  
Moderator: J. Britt Holbrook, Visiting Assistant Professor, School of Public Policy, Georgia Institute of Technology; and Member, AAAS Committee on Scientific Freedom and Responsibility
 
Kaye Husbands Fealing, Professor, Center for Science, Technology and Environmental Policy, Humphrey School of Public Affairs, University of Minnesota; Senior Study Director, National Academy of Sciences, Committee on National Statistics; and Member, AAAS Committee on Science, Engineering, and Public Policy
 
Gil Omenn, Director, Center for Computational Medicine and Bioinformatics, University of Michigan
 
Mike Taylor, Research Specialist, Elsevier Labs

 

How journals like Nature, Cell and Science are damaging science | Randy Schekman | Comment is free | The Guardian

Posted on by

These journals aggressively curate their brands, in ways more conducive to selling subscriptions than to stimulating the most important research. Like fashion designers who create limited-edition handbags or suits, they know scarcity stokes demand, so they artificially restrict the number of papers they accept. The exclusive brands are then marketed with a gimmick called \”impact factor\” – a score for each journal, measuring the number of times its papers are cited by subsequent research. Better papers, the theory goes, are cited more often, so better journals boast higher scores. Yet it is a deeply flawed measure, pursuing which has become an end in itself – and is as damaging to science as the bonus culture is to banking.

via How journals like Nature, Cell and Science are damaging science | Randy Schekman | Comment is free | The Guardian.

Thanks to my colleague Diana Hicks for pointing this out to me.

The last line of the quotation strikes me as the most interesting point, one that deserves further development. The steering effect of metrics is well known (Weingart 2005). There’s growing resistance to the Journal Impact Factor. Although the persuasive comparison between researchers and bankers is itself over the top, the last line suggests — at least to me — a better way to critique the reliance on the Journal Impact Factor, as well as other attempts to measure research. It’s a sort of reverse Kant with an Illichian flavor, which I will formulate as a principle here, provided that everyone promises to keep in mind my attitude toward principles.

Here is one formulation of the principle: Measure researchers only in ways that recognize them as autonomous agents, never merely as means to other ends.

Here is another: Never treat measures as ends in themselves.

Once measures, which are instruments to the core, take on a life of their own, we have crossed the line that Illich calls the second watershed. That the Journal Impact Factor has in fact crossed that line is the claim made in the quote, above, though not using Illich’s language. The question we should be asking is how researchers can manage measures, rather than how we can measure researchers in order to manage them.
_______________________________________________________

Peter Weingart. Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics Vol. 62, No. 1 (2005) 117-131.

Peter Higgs: I wouldn’t be productive enough for today’s academic system | Science | The Guardian

Posted on by

To what degree is quantity being substituted for quality in today’s research assessment exercises? This strikes me as a symptom of the overvaluation of efficiency.

Higgs said he became \”an embarrassment to the department when they did research assessment exercises\”. A message would go around the department saying: \”Please give a list of your recent publications.\” Higgs said: \”I would send back a statement: \’None.\’ \”

via Peter Higgs: I wouldn't be productive enough for today's academic system | Science | The Guardian.

Thanks to Lance Weihmuller for pointing me to the article.