Hints from Seasoned Study Section Members

June 2008 – Presentation delivered at NCCAM Grantsmanship Workshop

Linda Duffy, Ph.D.
Program Officer, NCCAM

What really goes on behind the study section doors?

You are in good company

Knowledge —

• "Nothing in life is to be feared, it is only to be understood".
• Madame Curie

What reviewers look for in investigator research

Feasible Research Plan — Specific aims, methods, and analytical plan are compellingly addressed.

Appropriate Methods — Proposes new approaches to test a solid hypothesis on a major problem.

Investigator Experience — Procedures and methods are appropriate and within the level of experience of the investigators.

Letters of Support — Provides strong letters of institutional commitment and collaborative support.

What reviewers look for in capacity-building

Science that thinks big and aims high.

Aim for the bull's eye

• Clarity — why the research is worth doing
• Precision — How the field will be moved forward

Innovation challenges

• Shifting Paradigms — Less experienced investigator may have more difficult time providing an innovative approach that shifts paradigm.
• Substantiate the Rationale — Failure to make a strong case for why investigator is challenging the existing paradigm often lacks substantiated data to support a proposed shift in the paradigm.
• Moving the Field — Either the means or the ends are innovative; both do not need to be but providing convincing argument for one or the other is the challenge.

New approaches to major problems

Innovation —

• "An innovation of armies can be resisted but not an idea whose time has come".
• Victor Hugo

Weigh preliminary data

• New Approaches — The more novel an idea or an approach tests an existing paradigm, the more necessary it is to have a solid hypothesis based on preliminary data that substantiates a claim.
• Investigator Laboratory — Evaluate how qualified the investigator is to handle the technologies and how much of the preliminary data is from the investigator's laboratory.

Competitive continuation applications

• Progress Reports — Summary of important findings to date should be included.
• Publications — Inclusion of published results [PMID] highlighting progress toward achieving specific aims.

Human subject protection training certification

• Confirm for key personnel that human subject protection training and education certification is completed.

Foreign institutions

• The relevance of the proposed research to the Institute or Center mission is weighed in the balance for determining need for the research.
• If similar work is being done in the U.S., it can lower reviewer enthusiasm.

Overlaps — testing the waters

• Scientific Overlap — Note whether another organization or NIH grant is supporting a part of the research plan.
• Commitment Overlap — Check that the committed effort of investigators does not exceed 100%.
• Budgetary Overlap — Determine that there is not a request for resources paid for by another source.

What makes a good application?

Guiding Vision

What makes a good application?

Strategic Decisions

Putting it together

• Responsive to RFA/PA — Meets objectives, in a well-organized research plan.
• Significant Research — Problem is significant to the field and is likely to have an impact on public health.
• Innovative — Modest to highly novel, and the approach is well supported
• Strong Institutional Commitment — Resources and environment are adequate with strong institutional support for the research.
• Qualified Investigators — The investigator team is the ideal candidate to conduct the research and is likely to accomplish the specific aims.
• Strong Mentoring Plan — Career development plan high likelihood to achieve investigator success.

How to make a study section member smile?

• Impact — Knowing an application will make an original and important contribution.
• Feasibility — Seeing conceptual design, methods, and analytical plan tightly integrated.
• Investigator Environment — Recognizing that the institutional commitment, mentoring plan, research plan and collaboration create synergy.
• Innovation — Reading exciting, novel ideas that are well supported.

What makes a bad application?

Scope is Too Narrow

What reduces enthusiasm

• Premature Research Plan — One or two aims with very narrow scope that lack substantiated rationale.
• Overambitious Scope — Plan does not realistically fit the request for time and resources.
• Unfeasible Approach — Problem is more complicated and investigators lack background for a high likelihood of success.
• No Alternative Strategies — Potential obstacles and pitfalls are not considered, if success of initial experimental set does not succeed as expected.
• Inappropriate Model System — The model organism proposed is inappropriate.
• Poor Control Selection — Controls for human studies are inadequately addressed.
• Flawed Analytical Plan — Insufficient consideration is given to the sample size, power, and other statistical needs.

What impacts an unscored application

• Overambitious amount of work and unrealistic timeline
• Inadequate protection of human subjects, laboratory staff, or animal safety
• Failure to acknowledge key literature and factual inaccuracy

What can happen to an application

• Streamlined — unanimously judged to be roughly in the bottom half of applications. These are not discussed; the reviewer critiques serve as the summary statement.
• Deferred — postponed if unable to determine an application's scientific merit because important information is missing.
• Not Recommended for Funding Consideration (NRFC) — used for applications that lack significance and substantial scientific merit or have serious hazards or ethical issues.

How to make a study section member cringe?

Failure to proofread

Blurred Vision

• Excessive Detail — The more information there is, the more dense the reading, and more likely reviewers will lose interest.
• Lack of Organization — Enthusiasm is reduced for unlabelled, disorganized sections, appendices that are overly dense and unformatted, and pages that do not match the table of contents.
• Not Following the Format Rules — Enthusiasm is further reduced when the application exceeds page limit, reduced font size, and spacing are used.
• Too Many Acronyms — Medical acronyms are not clearly understood, and excessive abbreviating throughout text.

What reviewers look for in budgets

• Scope of Budget — Budget request as compared to the investigator career level, grant type, and research goals.
• Budget Matches Text — Credibility is lowered for equipment requests in a budget that are stated in the research plan as available.
• Realistic Timeframe — Specific aims are considered as to whether they are achievable within the budget and timeframe planned.
• Roles and Effort — Key personnel roles and time-effort need to be detailed in the budget justification.

Life is a Journey

Excellence —

• "When love and skill work together, expect a masterpiece".
• John Ruskin

Note: PDF files requires a viewer such as the free Adobe Reader.