Nature | Comment
Research efficiency: Perverse incentives
- Journal name:
- Nature
- Volume:
- 484,
- Pages:
- 29–31
- Date published:
- DOI:
- doi:10.1038/484029a
- Published online
Counterproductive financial incentives divert time and resources from the scientific enterprise. We should spend the money more wisely, says Paula Stephan.
Subject terms:
Summary
- Science is full of incentives that encourage bad financial choices, such as expanding labs and hiring too many temporary scientists.
- These incentives hurt both individual scientists and society as a whole, which gets minimal return on its investment when someone is trained for a field with no career prospects.
- The way forward is to fix incentives that are damaging the system, by considering their true social and personal cost.
Scientists may portray themselves as not being motivated by money, but they and the institutions where they work respond in spades to financial opportunities. Incentives that encourage people to make one decision instead of another for monetary reasons play an important part in science. This is good news if the incentives are right. But if they are not, they can cause considerable damage to the scientific enterprise.
For instance, cash incentives adopted by countries such as China, South Korea and Turkey encourage local scientists to submit papers to high-end journals despite the low probability of success. These payments have achieved little more than overloading reviewers, taking them away from their work, and have increased submissions by the three countries to the journal Science by 46% in recent years, with no corresponding increase in the number of publications1.
ILLUSTRATION BY VIKTOR KOEN
Sadly, science is full of incentives gone awry. Look no further than expanding PhD programmes that produce graduates with almost no career prospects, or the growth of lab space with no apparent increase in productivity.
The economic rules behind science were written without much consideration for unintended consequences, but such consequences abound because people and institutions are so responsive to incentives. And in the current economic climate, no one can afford to waste time or resources. In a world of tight budgets, getting the incentives right is more important than ever.
Bad directions
Consider the financial calculations that encourage universities to hire a series of postdocs rather than staff scientists. Postdocs earn around half to two-thirds of a staff scientist's salary. They are young, have fresh perspectives and new ideas and are temporary, so can be let go when budgets decline2. But, in reality, postdocs are not cheap: substantial resources — both their own and society's — have been invested in training them.
If a postdoc doesn't get a research job, taxpayers do not get a return on their investment. Neither does the postdoc: someone who did not go to graduate school and entered the labour market in 2001 was earning about US$58,000 in 2008; a first-year postdoc who started graduate school in the United States in 2001 was making around $37,000 in 2008 on graduation3. During a three-year postdoc position, a scientist gives up more than $60,000 on average in return for highly uncertain job prospects. And many postdocs will not get a research job. There are few faculty openings, and limited numbers of research positions in government and industry. So even if individual postdocs cost less, from a societal perspective they can be expensive.
Equally harmful are rules that encourage scientists to support graduate students on a research assistantship (RA) rather than on a training grant, despite evidence that the latter produces better outcomes. Part of the reason is that RA funding comes with an additional amount to cover the university's overhead, or indirect rate, which may be as high as 50%. For those on training grants from the US National Institutes of Health (NIH) in Bethesda, Maryland, that amount is capped at 8%. This difference easily translates into an institution getting at least $12,000 more for every RA-supported student. Other considerations affect the choice of RAs over training grants, too — RAs are under the direct control of principal investigators, whereas graduate students on training grants are less so.
However, training grants are arguably better for scientists in the long term. Importantly, they give departments the incentive to provide a high-quality training experience, because renewals for training-grant awards are evaluated on the quality of the PhD experience and placement outcomes. By contrast, scientists who support students on research grants are not required on renewal to disclose where graduates end up being placed. Some principal investigators collect this information, but departments typically do not — my informal survey of 45 science departments found only two that were able to report where their students had been placed. Without this knowledge, prospective students will not be able to judge whether a lab is a good place to begin a successful science career.
The growth of labs is another result of incentives. Bigger is seen as better: more funding, more papers, more citations and more trainees — regardless of whether the market can sustain their employment. Some institutions pay bonuses to faculty members on the basis of the amount of external funding they receive4. But, again, too many trainees creates a glut of people who will not find suitable jobs. It would have been more efficient for both the students and society to steer them in a different direction. And big labs can be wasteful — an analysis by the US National Institute of General Medical Sciences in Bethesda, Maryland, found that an increase in funding is not associated with a substantial increase in output when measured by the number of grant-linked publications5.
Other economic incentives indirectly render the scientific process less efficient — such as the tendency of scientists to avoid risk by submitting to funding organizations only those proposals that they consider 'sure bets'. This tendency comes from the need for faculty members to obtain grants to support their salary, the emphasis on preliminary data in grant applications and the difficulty of obtaining funding in today's climate. If most scientists are risk-averse, there is little chance that transformative research will occur, leading to significant returns from investments in research and development. Funding bodies sometimes give money specifically for field-changing research, but not nearly enough — Pioneer grants from the NIH fund fewer than 1% of applicants.
“The building boom is now costing the scientific enterprise by creating space that cannot be paid for.”
In the European Union, there are strong incentives for researchers to team up with colleagues in other countries. This is because most funding opportunities under the various research Framework programmes require consortia to be made up of at least three entities in three different European countries. No collaboration, no grant. Is this a good use of resources? Although there is evidence that collaboration leads to better research, I do not know of any that supports the idea that those collaborators should come from different countries. The extra costs of coordination — organizing the work, travel, meetings and so on— can be large relative to the money being invested in research.
Universities are also driven by incentives. By hiring faculty members on 'soft' money, with grants providing the salary, the institutions bear almost none of the risks. Furthermore, universities prefer to put up a new building or invest millions in remodelling existing lab space rather than house scientists in older buildings that they already own. Why? One reason is that debt can be an accounting asset. A US government accounting rule called A21 means that the more debt universities have from construction, the more they can add to grants for overhead costs. If a university borrows $100 million to build a new facility and pays 4% interest, it can increase its indirect rate by including the $4-million interest payment in the calculation. The building binge is further fuelled by competition among universities: recruiting senior faculty members requires space, and lots of it.
What is so bad about institutions putting up new research facilities? The answer lies in what economists call 'incidence' — who eventually pays. Before the global financial crisis, universities had hoped to recoup the money through increased indirect costs and through the 'buy-out' money they receive from funding agencies to cover the salaries of permanent faculty members working on grants. But now that grants are harder to get, that money isn't coming in. Unless states and private institutions default, someone will have to pay the bonds. The money is likely to come from the physical sciences, the humanities and social sciences, as well as cutbacks in hiring across departments. In short, the building boom is now costing the scientific enterprise by creating excess space that cannot be paid for.
Fix what's broken
The way forward is to alter these damaging incentives. The scientific enterprise should cut back on the demand for graduate research assistants by establishing more research institutes that are not focused on the production of PhDs, such as the Howard Hughes Medical Institute's Janelia Farm campus in Ashburn, Virginia. Research institutes, by producing fewer PhDs, lead to a better balance between supply and the limited number of research jobs. Abstinence, after all, is the most effective form of birth control.
In addition, we should consider ways of making graduate students and postdocs more costly to universities, to discourage their overuse and reflect their social cost. One possibility is to 'tax' the two positions, making them more expensive relative to other staff types, thereby providing an incentive to employ permanent rather than temporary staff. Principal investigators and their departments should also be required to report placement data online as part of all research-grant applications. This would allow society to monitor the return on its investment, and students to assess job outcomes.
SOURCE: US NATL SCI. FOUND.
Training grants should be made more economically attractive. And rules should be altered to limit the amount of interest payments universities can include when calculating indirect rates, and the amount of faculty members' salaries that can be charged to grants, thereby dulling the incentive to hire people for soft-money positions. Shifting evaluations from projects to people, and de-emphasizing the importance of metrics in hiring and promotion, could encourage scientists to work on riskier projects6.
Many of the problems now faced by science accelerated when biomedical funding started to increase steeply. For instance, the doubling of the NIH budget from 1998 to 2003 triggered universities to hire more people and build more buildings, while scientists increased the number of grants they submitted and the size of their labs (see 'Biology's research footprint'). Now, this biomedical machine needs increasing amounts of money to sustain itself — larger labs need more grants, which leads to lower success rates, with calls for more funding.
Biomedical research has done much to contribute to increased life expectancy. But it seems likely that diminishing returns have set in. New drugs are slower in coming to market and there was a less than stellar increase in US publications associated with the NIH doubling7. Moreover, many of the breakthroughs that have contributed to better health outcomes have come from other fields of science — such as the laser and magnetic resonance imaging. Funds for the physical sciences in the United States (in terms of the percentage of federal research funding) are close to a 35-year low. Perhaps it is time for deans in the biomedical sciences to rent some of that excess space to their colleagues in chemistry and physics.
Report this comment #40972
I'm so happy that someone finally came out and said that most science PhDs have no real career prospects, and probably won't wind up in research. Between my undergrad, grad, and post-doc training I've given over a decade of my life to science. My reward is looking like it's going to be a toss up between a scientific writing position, teaching English in a foreign country, or being an unemployed bum living with my parents.
I fully support the idea of cutting off funding for grad students and post-docs from individual research grants and moving them solely to training grants that force the institution to be accountable for what they're producing. In order to stay competitive for training grants, institutions should be required to actually train their students and prepare them for a variety of careers, not just R01 chasing academic. Mentors and departments who fail to do this should not be allowed to have trainees.
Here's several ideas to "tax" trainee positions. Make institutions provide better pay to grad students and post-docs. Grad student stipends could increase each year as their skills and productivity increases. Grad students should not be used for teaching scut work – that means no proctoring exams, making copies, playing gopher to the scantron office, or grading essay questions. If the department wants grad students or post-docs to do these kind of menial tasks they should be required to pay them extra. Post-docs should not be making less than some entry level BA in marketing. And how about providing actual retirement and health benefits?
I could go on forever on this topic, but I'll end it here.
Report this comment #40987
I hope this paper stimulates a lot of discussion and leads to beneficial changes. I agree with many of the specifics, but not all.
Stephan writes that "If a postdoc doesn?t get a research job, taxpayers do not get a return on their investment." For "get a research job", I would substitute do something worthwhile (during the postdoc and subsequently) that wouldn't have been possible without the postdoctoral experience." Depending on the nature of that experience, subsequent nonresearch jobs requiring advanced scientific training could range from developing better crops for a seed company to upgrading a forensics lab to leading science-themed tourism to teaching in traditional and nontraditional ways (broadly defined to include internet-based education and agricultural extension, for example).
I agree with Stephan's main point, however. Individual faculty members deciding whether to support graduate students, postdocs, or staff will decide mainly based on individual incentives — which choice is most likely to lead to the high-profile publications needed for promotion and to get future grants? -- which may not always correspond to the greatest long-term benefits to society. And, for research supported by tax money, that should be the criterion.
Note that guaranteeing everyone who earns a PhD the kind of job he or she was hoping for probably wouldn't maximize societal benefits either. How would the quality of university faculty change if there were only one applicant for each opening?
Report this comment #41027
What about people who are doing research while being in the country on temporary visa and having no other options of income, also with a dependent family?
Report this comment #41052
I agree with most, if not all, of Stephans' point. I read her book a few weeks ago, and I highly recommend that all scientists and administrators read the entire book. I also think that all prospective graduate students should be required to read the book and this commentary, so that they know what they are getting themselves into when deciding to go to graduate school, at least in America.
Report this comment #42018
Regarding "cash incentives ... to submit papers to high-end journals despite the low probability of success. These payments have achieved little more than overloading reviewers, taking them away from their work, and have increased submissions by the three countries to the journal Science by 46% in recent years":
1. One should note that for smaller players it is hard to get published in the "high-end" journals. One of the reasons may be the lack of network – not the insufficient quality of the research work. Note that the affiliations of authors are often revealed to reviewers.
2. How to quantify the quality of the review process? Submission and acceptance rate statistics does not tell full story.
3. Although in many cases, submissions from less known institutions in the above-mentioned countries may be of insufficient quality to be published in the "high-end" journals, reviewers are probably less inclined to support submissions from institutions they have never heard of. This is related to the question: how useful is the anonymous peer review process?
4. Even though the acceptance rates for bigger players (countries who have been successful in the R&D in the past century) are probably higher (does anybody have data to comare?), there are a number of articles published in the "high-end" journals – written by the scientists from reputed universities in these countries – that do not show a significant breakthrough in their respective field. I guess many of us can quote some examples from various research fields.
5. Many scientists from the "smaller player" countries do high quality research but are not yet capable to present their work in English language in an attractive way, and for a broad audience.
6. Even if the number of submissions raised, it is probably a temporary effect. In the end, scientists will probably weigh the pros and cons of submitting their work to the "high-end" journals such as Science.
7. If a paper is not accepted by any of the "high-end" journals such as Science, the funds can be diverted to another purpose, for example,improvement of infrastructure. But please do not draw a conclusion, that turning down a good paper will profit the author or their institution more!
8. Do you really believe that reviewing some of the manuscripts from the countries mentioned above will affect the reviewers to great extent? Low-quality submissions can readily be screened out by the editors, as a part of their daily work.
9. If a manuscript to review is of low quality, then the reviewers will not need to spend too much time to write a negative report; they are experts in the field, so they can quickly point out the flaws.
10. This is not about any "perverse incentive" of the countries mentioned above, it is about changing the way to publish scientific results, in a more transparent and egalitarian manner. Why can't every researcher publish their breakthrough work online in specially designed community portals/databases? Perhaps this would enhance co-operation, and evaluating scientific output in a fair way. The numbers of "Likes" could be a measure to grade the researchers' performance.
11. Authors fill up journal pages for free (or sometimes even need to pay a publication fee!), and the publishers typically retain the copyright. Somebody has to compensate the efforts made by the authors. If the publishers cannot do that, the employers of the authors may want to do that.
...and as one reviewer ever ended their vague report: "etc. etc.".
To sum up, although the cash-for-paper may have its drawbacks, there are certainly other issues that have to be dealt with in the first place.
Report this comment #42059
It's an interesting idea to limit the no.s of grad students and postdocs enrolled or hired, but it's somewhat to naive to think that this would have no impact either on research output or the quality of academic faculty hires/ industry managers. The culling process at each stage of a science career, from undergrad to graduate to postdoc to faculty (or industry) is an extraordinarily effective means of selecting the very best people. Reducing the pool on which that selection process acts doesn't magically remove only the least bright and poorer performers, leaving just the geniuses behind; it reduces the variety and quality of the pool evenly as people from across the performance spectrum decide against entering that career path.
The idea to limit the no.s of grad students and postdocs enrolled or hired is also mistakenly predicated on the assumption that the increased intake, without corresponding increase in faculty positions, means that ever increasing no.s of PhD holders make no valid use of their qualifications. This is patently absurd and simply reflects the predominant "academia only" bias with which the attainment of a PhD is viewed. Considerably numbers of industries and professions related to the sciences require a PhD and often postdoctoral experience, including but not limited to; private industry, scientific publishing, advocacy groups, consultancies, sport involving engineering or training and so on. The increase in graduate numbers increases a country's knowledge base, expertise, R+D capability and so on. Restricting graduate numbers, in the name of better fitting PhD numbers to faculty positions, is disappointingly blinkered.