In April 2013, some of the original signers of DORA wrote to executives at Thomson Reuters to suggest ways in which it might improve its bibliometric offerings. Suggestions included replacing the flawed and frequently misused two-year Journal Impact Factor (JIF) with separate JIFs for the citable reviews and for the primary research article content of a journal; providing more transparency in Thomson Reuters’ calculation of JIFs; and publishing the median value of citations per citable article in addition to the JIFs. Thomson Reuters acknowledged receipt of the letter and said, “We are carefully reviewing all the points raised and will respond as soon as possible.” After more than a year, they have not responded or addressed the concerns of the signers.
April 16, 2013
Marian Hollingsworth, Director, Publisher Relations
Marie McVeigh, Director, JCR and Bibliographic Policy
Thomson Reuters
1500 Spring Garden Street
Philadelphia, PA 19130
cc: Jonathan Adams, Director of Research Evaluation, Thomson Reuters
James Testa, Thomson Reuters
Re: Proposal for Thomson Reuters to Modify the Journal Impact Factor
Dear Ms. Hollingsworth and Ms. McVeigh:
Thomson Reuters’ two-year “Journal Impact Factor” (JIF) is a dominant bibliometric indicator employed by libraries, funders, research institutions, and researchers to assess the value of journals. As you know, there are a number of inherent limitations of the two-year JIF as a single metric, in particular when applied to research assessment, and Thomson Reuters has in fact launched a number of alternative metrics, such as the five-year JIF and the Eigenfactor. This is laudable.
However, since the two-year JIF remains the single metric most widely used by the research community for assessing journals, we write to make the case for correcting three inherent deficiencies in this metric: the use of a single JIF for reviews and primary research article content of journals, the lack of transparency about how the JIF is calculated, and the lack of a median value for citations per citable article.
The two-year JIF is widely misunderstood and therefore misused by the scientific community, especially in the assessment of scientists. Rather than being judged on the importance of their research papers, individual scientists are frequently judged instead by the JIF of the journals in which they have published. The JIF was never intended to be applied to the assessment of individual scientists, and the misuse of the JIF by the research community is not the fault of Thomson Reuters. Nevertheless, we believe that the organization has an obligation to make the JIF as robust, fair, and transparent as possible.
A group of editors of highly regarded journals that publish in the biomedical sciences met recently to discuss limitations of the two-year JIF and how the JIF and other metrics might be better used by the scientific community. We hope to educate the scientific community regarding the misuse of journal-specific metrics in evaluating science and scientists. At the same time, we hope that such metrics can be improved to better reflect the importance of a given journal to its field. The undersigned herewith make three suggestions to render the JIF more meaningful.
We propose that:
- Thomson Reuters replace the current JIF with separate JIFs for the citable reviews and for the primary research article content of a journal. As you know, reviews and primary research papers have rather different citation dynamics, and this can dramatically influence the overall JIF for a journal. A number of factors contribute to this difference. One is the imposition by many journals of limits to the number of allowable citations in a paper (or character/word limits that de facto limit the number of citations), which encourages authors to cite reviews as a means to bundle references to multiple papers. Another is the rapidly growing scientific publication rate, which has made it more difficult for authors to become familiar with all of the relevant primary research papers. This has led to an inflation of citations to reviews and, consequently, the loss of citations to primary research papers. As a result, the contribution of the authors of the primary papers who actually made the primary scientific discoveries is devalued, as is the rating of journals that focus mainly on publishing primary literature relative to those that publish many reviews. The publication of separate JIFs for reviews and primary research content will immediately remove the incentive for journals to manipulate the JIF by publishing more reviews. It will also ensure that credit is given where credit is due.
- Thomson Reuters provide more transparency in its calculation of these JIFs. In particular, we recommend the publication of the percentage of total citations that are to content classed as non-citable by Thomson Reuters. Although Thomson Reuters openly describes the method used to calculate the JIF from the number of citations to a paper, Thomson Reuters is less clear about how citations are counted—particularly what types of articles are counted as “citable” for a given journal (the denominator). Greater transparency would better allow journals to understand the trends in their JIF, discourage journals from manipulating or “gaming” the IF by making it more apparent that they are doing so, and provide valid means to compare the JIF to different bibliometric measures.
- Thomson Reuters publish the median value of citations per citable article in addition to the JIFs. The current JIF is a collective ratio that can be skewed by a paper that garners a disproportionate number of citations. Although we realize that there are also limitations to the median as a measure of distribution, the combination of JIF and article median would provide a more transparent indication of the distribution of citation numbers among citable articles than the JIF alone.
In our view, these proposed changes to the JIF are essential to ensure that the impact factor is a more useful measure of journal significance. We hope that you agree and look forward to your response.
Yours sincerely,
Peter Adams, co-Editor-in-Chief of Aging Cell, Professor, University of Glasgow
Elizabeth M. Adler, Executive Editor of The Journal of General Physiology
Sharon Ahmad, Executive Editor of The Journal of Cell Science
Bruce Alberts, Editor-in-Chief of Science Magazine
Brenda Andrews, Editor-in-Chief of G3: Genes|Genomes|Genetics; Charles H. Best Chair of Medical Research, Professor and Chair, Banting & Best Department of Medical Research, Director, The Donnelly Centre for Cellular and Biomolecular Research, University of Toronto
Adam Antebi, co-Editor-in-Chief of Aging Cell; Chair, Max Planck Institute for Biology of Ageing
Stefano Bertuzzi, Executive Director, American Society for Cell Biology
David Botstein, Founding Editor-in-Chief of Molecular Biology of the Cell; Director Lewis-Sigler Institute for Integrative Genomics, Princeton University
Philip Campbell, Editor-in-Chief, Nature
Don W. Cleveland, President, American Society for Cell Biology; Professor, University of California, San Diego
Ana Maria Cuervo, co-Editor-in-Chief of Aging Cell; Professor, Albert Einstein College of Medicine
Stefanie Dimmeler, Chief Editor of EMBO Molecular Medicine; Professor of Experimental Medicine and Director of the Institute of Cardiovascular Regeneration, Centre for Molecular Medicine at the University of Frankfurt, Germany
David G. Drubin, Editor-in-Chief, Molecular Biology of the Cell; Professor, University of California, Berkeley
Adam Fagen, Executive Director, Genetics Society of America
László Fésüs, chairman of Publications Committee, Federation of European Biochemical Societies
Christian Gericker, Associate Editor, BMC Health Services Research; CEO, The Wesley Research Institute
Peter Gunning, President, The American Society for Biochemistry and Molecular Biology, Editor-In-Chief, BioArchitecture
Lisa Hannan, Managing Editor of Traffic
John Inglis, Executive Director and Publisher, Cold Spring Harbor Laboratory Press
Reinhard Jahn, Department of Neurobiology, MPI for Biophysical Chemistry; EMBO Publications Advisory Committee (chair); EMBL Scientific Advisory Board (vice chair); Dean, Göttingen Graduate School for Neurosciences, Biophysics, and MolecularBiosciences
David James, Director, Diabetes and Obesity Program, Garvan Institute of Medical Research
Mark Johnston, Editor-in-Chief of GENETICS;Professor and Chair, Department of Biochemistry and Molecular Genetics, the University of Colorado School of Medicine
Brian Kennedy, co-Editor-in-Chief of Aging Cell; President and CEO, Buck Institute for Aging Research
Véronique Kiermer, Executive Editor and Head of Researcher Services, Nature Publishing Group
W. Mark Leader, Publications Director, American Society for Cell Biology
Thomas Lemberger, Chief Editor of Molecular Systems Biology; and Deputy Head of Scientific Publications, EMBO
Michael Lynch, President, Genetics Society of America; Distinguished Professor of Biology, Indiana University at Bloomington
Michael S. Marks, co-Editor of Traffic; Professor, University of Pennsylvania
Mark Marsh, co-Editor of Traffic; Director of The Medical Research Council Laboratory for Molecular Cell Biology, University College London
Tom Misteli, Editor-in-Chief of The Journal of Cell Biology
Timothy W. Nilsen, Editor-in-Chief of RNA
Kathryn North, Director, Murdoch Childrens Research Institute
Olivier Pourquie, Université de Strasbourg
Bernd Pulverer, Chief Editor of The EMBO Journal; and Head of Scientific Publications, EMBO
Jordan Raff, President of the British Society for Cell Biology and Editor-in-Chief of Biology Open
Roger Reddel, Lorimer Dods Professor and Director, Children’s Medical Research Institute
Mike Rossner, Executive Director, The Rockefeller University Press
John Sedivy, co-Editor-in-Chief of Aging Cell; Professor, Brown University
Robert Shepherd, Director, Bionics Institute, University of Melbourne
Tom Stevens, co-Editor of Traffic; Professor, University of Oregon
Marlowe Tessmer, Senior Editor of The Journal of Experimental Medicine
Michael Way, Editor-in-Chief of The Journal of Cell Science
Liz Williams, Executive Editor of The Journal of Cell Biology
