Contents

---

Executive Summary

Purpose and Scope

Findings

Conclusion

Appendix A. Universities Visited

Appendix B. Method

Appendix C. Interview Guides

Suggested Citation, Acknowledgments

Executive Summary

Scientists and engineers ordinarily publish their research results in peer-reviewed journal articles. The number of these articles is an indicator of research output, although an admittedly imperfect one. In recent years, international use of this and related indicators has become widespread as countries have sought to assess their relative performance in science and engineering (S&E) research.

This report summarizes the views of experienced observers and practitioners in research universities about how the worlds of academic S&E research and publication changed between 1988 and 2003. It is part of a larger study by the National Science Foundation, Division of Science Resources Statistics (SRS), of changing patterns and trends in U.S. production of scientific articles since the late 1980s. The study was prompted by evidence that the growth in the number of U.S. articles, which had continued for more than two decades, began to slow in the 1990s even though research and development funds, research personnel, and similar research inputs continued to grow. At the same time, comparable evidence indicated that growth in article counts from leading research-producing countries in Europe and Asia continued unabated. The core of the larger study is a quantitative examination of patterns and trends in article production in the 15-year period between 1988 and 2003.

Data were gathered during visits to nine universities in the upper tier of the American academic research universe, where meetings were held with experienced researchers and research administrators across the spectrum of S&E disciplines. By visiting universities and talking with informants knowledgeable about academic research, SRS sought to better understand the changing circumstances that might affect the publication patterns observed in the quantitative data. Because the purpose in visiting universities was essentially exploratory, interviews and meetings ranged widely, probing changes and continuities in several domains, including how research is done, how the publication process works, and what activities universities foster and value. Perceived changes in research and publication outside U.S. universities, both internationally and in the U.S. nonacademic sector, were also explored.

Findings showed that peer-reviewed articles remain the major vehicle by which research findings are validated and scientists obtain credit for their contributions. Despite the rise of new forms of research output associated with advances in information technology, such as databases, software programs, and contributions to electronic archives, and new ways to disseminate findings electronically, there was little evidence to suggest that the validity of article and citation counts as output indicators was diminishing. According to those interviewed, data on article counts are unlikely to mask or distort real changes in scholarly output, except, possibly, in computer sciences. If U.S. researchers figure less prominently in the journal literature, the reason does not appear to be because they are reporting their findings in ways that bypass the journals.

Those interviewed consistently reported that the research done in other developed countries and in several emerging Asian economies is getting better and becoming more abundant. In their view, improved capacity overseas is more likely to account for the increased share of S&E papers from foreign institutions than changes in what Americans have been doing. In an expanding literature, they see a continuing, even growing, American presence, but more marked growth occurring in other countries.

Advances in communication have made the international scientific literature more accessible to researchers in other countries. In this regard, advances in electronic communication loom large. As potential contributors to the literature, researchers can take advantage of improved electronic communication to collaborate more easily with distant colleagues and submit papers online. As readers, they can receive papers from colleagues via e-mail, find information in electronic archives and databases, and access scientific communications that cannot be found in a local university library. In addition to electronic communication, increased capacity worldwide to communicate in a common scientific language, English, has also played a role.

As the largest and most influential producer of scientific articles in the world and a nation whose native language is also the dominant language of science, the United States was already at the center of the worldwide system of scientific communication before these advances occurred. Thus, journals were already highly accessible to U.S. researchers, both as contributors and as readers, at the outset of the period studied. Improvements in communication may have had a greater effect on the ability of researchers elsewhere in the world—especially those in nations or at institutions that were not prominent in research in the late 1980s—to keep up with their fields, produce research of a reasonable quality, and report their research in journals with a wider audience and a greater impact.

Institutional differences between the United States and other major research-producing countries may also be affecting article counts. The U.S. researchers interviewed for this study perceive their universities and funding agencies as less attuned to quantitative measures of output and impact than their institutional counterparts in other countries. As a result, U.S. researchers may be less concerned with producing scholarly output in ways that score well on these measures. Study informants stressed the role of expert judgment in maintaining the commitment to quality in the U.S. system and saw the system as being somewhat more oriented toward quality than in the past. They portrayed pressures toward scientific productivity as increasing, if anything, but as directed more toward enhanced quality than toward greater quantities of output. In contrast, many of the U.S. researchers interviewed saw other countries' efforts to improve research during the period under study as being increasingly driven by quantitative measures.

The study's findings provide little support for the idea that competing institutional demands are diverting faculty from research and publication. For the most part, informants said that neither teaching nor commercial activities were absorbing time that in the past would have been devoted to research and writing. Although some saw increased university concern about good teaching, and all agreed that institutional support for commercial activity was growing, faculty continue to believe that research is clearly the institutional concern that mattered most in shaping their behavior. It is possible, of course, that activities that compete with research for faculty time and attention, especially commercialization-related activities, have adverse effects on publication outputs that researchers themselves do not fully appreciate.

One of the most striking recent changes in how research is done has been the movement toward more collaborative work, especially interdisciplinary and interinstitutional collaboration. Study data suggest that this trend can have either of two opposite effects on publication output. Insofar as it involves "complementary" collaborations that increase research output via a more rational division of labor, it should generate increased numbers of publications. However, insofar as this trend involves "integrative" collaborations that require extensive communication to synthesize different perspectives on a problem into a coherent piece of research, more people, money, and time may be required to produce a publishable article. It is possible that growth in publication output has slowed as a result of a movement toward integrative collaborations. Some informants suggested that successful integrative collaborations have had disproportionate impact on their fields and that the United States has been in the forefront of movement toward this type of collaboration. If U.S. researchers, compared to researchers in other countries, had been more rapidly increasing their investment of time and resources in this type of collaboration, this might help explain the change in article counts.

The increasing time and effort that U.S. researchers say they devote to securing funding may also adversely affect U.S. article counts. Some researchers said that in the United States, competitive mechanisms intended to spur productivity sometimes and increasingly had the opposite effect, even as other countries profited from introducing more such mechanisms. Similarly, although some informants indicated that regulatory and governance burdens are increasing at U.S. universities, there was little to indicate whether the situation in other countries is different in this regard.