Definitions of Criteria and Considerations for BRP Critiques

[Return to Review Guidelines Page]

Overall Impact. After considering all of the review criteria, briefly summarize the significant strengths and weaknesses of the application and state the likelihood of the project to exert a sustained powerful influence on the field.

1. Significance. Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?

2. Investigator(s). Are the PD/PIs, collaborators, and other researchers well suited to the project? If Early Stage Investigators or New Investigators, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project?

3. Innovation. Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?

4. Approach. Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed?

If the project involves clinical research, are the plans for 1) protection of human subjects from research risks, and 2) inclusion of minorities and members of both sexes/genders, as well as the inclusion of children, justified in terms of the scientific goals and research strategy proposed?

5. Environment. Will the scientific environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements?

Partnership and Leadership

Is the proposed partnership adequate for the research? Is there evidence that the partnership will be effectively managed by the PI or project manager? Is the partnership strategy well planned and documented? Is there evidence that the partners from academia or industry can work together effectively, have an impact on achieving the research goals, and disseminate the developed technology?

Protections for Human Subjects. For research that involves human subjects but does not involve one of the six categories of research that are exempt under 45 CFR Part 46 (as described in Human Subjects Protection and Inclusion), reviewers are asked to evaluate the justification for involvement of human subjects and the proposed protections from research risk relating to their participation according to the following five review criteria: 1) risk to subjects, 2) adequacy of protection against risks, 3) potential benefits to the subjects and others, 4) importance of the knowledge to be gained, and 5) data and safety monitoring for clinical trials. If all of the criteria are adequately addressed, and there are no concerns, write "Acceptable Risks and/or Adequate Protections." A brief explanation is advisable. If one or more criteria are inadequately addressed, write, "Unacceptable Risks and/or Inadequate Protections" and document the actual or potential issues that create the human subjects concern. Also, if a clinical trial is proposed, evaluate the Data and Safety Monitoring Plan. (If the plan is absent, notify the SRO immediately to determine if the application should be withdrawn.) Indicate if the plan is "Acceptable" or "Unacceptable", and, if unacceptable, explain why it is unacceptable.

For research that involves human subjects and meets the criteria for one or more of the six categories of research that are exempt, evaluate: 1) the justification for the exemption, 2) human subjects involvement and characteristics, and 3) sources of materials. If the claimed exemption is not justified, indicate “Unacceptable”, and, if unacceptable, explain why it is unacceptable.

NOTE: To the degree that acceptability or unacceptability affects the investigator's approach to the proposed research, such comments should appear under "Approach" in the five major review criteria above, and should be factored into the score as appropriate.

For additional information to assist you in making these determinations, please refer to Human Subjects Protection and Inclusion and Human Subjects Worksheet for Comments.

Inclusion of Women, Minorities, and Children. When the proposed project involves clinical research, the committee will evaluate the proposed plans for inclusion of minorities and members of both genders, as well as the inclusion of children.

Public Law 103-43 requires that women and minorities must be included in all NIH-supported clinical research projects involving human subjects unless a clear and compelling rationale establishes that inclusion is inappropriate with respect to the health of the subjects or the purpose of the research. NIH requires that children (individuals under the age of 21) of all ages be involved in all human subjects research supported by the NIH unless there are scientific or ethical reasons for excluding them. Each project involving human subjects must be assigned a code using the categories "1" to "5" below. Category 5 for minority representation in the project means that only foreign subjects are in the study population (no U.S. subjects). If the study uses both then use codes 1 thru 4. Examine whether the minority and gender characteristics of the sample are scientifically acceptable, consistent with the aims of the project, and comply with NIH policy. For each category, determine if the proposed subject recruitment targets are "A" (acceptable) or "U" (unacceptable). If you rate the sample as "U", consider this feature a weakness in the research design and reflect it in the overall score. Explain the reasons for the recommended codes; this is particularly critical for any item coded "U".

Gender Inclusion Code

Minority Inclusion Code

Children Inclusion Code

G1 = Both genders            

M1 = Minority and nonminority      

C1 = Children and adults

G2 = Only women

M2 = Only minority

C2 = Only children

G3 = Only men         

M3 = Only nonminority

C3 = No children included

G4 = Gender composition unknown

M4 = Minority composition unknown

C4 = Representation of children unknown

 

M5 = Only foreign subjects

 


NOTE: To the degree that acceptability or unacceptability affects the investigator's approach to the proposed research, such comments should appear under "Approach" in the five major review criteria above, and should be factored into the score as appropriate.

For additional information to assist you in making these determinations, please refer to Human Subjects Protection and Inclusion and Human Subjects Worksheet for Comments.

Vertebrate Animals. The committee will evaluate the involvement of live vertebrate animals as part of the scientific assessment according to the following five points: 1) proposed use of the animals, and species, strains, ages, sex, and numbers to be used; 2) justifications for the use of animals and for the appropriateness of the species and numbers proposed; 3) adequacy of veterinary care; 4) procedures for limiting discomfort, distress, pain and injury to that which is unavoidable in the conduct of scientifically sound research including the use of analgesic, anesthetic, and tranquilizing drugs and/or comfortable restraining devices; and 5) methods of euthanasia and reason for selection if not consistent with the AVMA Guidelines on Euthanasia. For additional information to assist you in determining if the Vertebrate Animals section is “Acceptable” or “Unacceptable”, please refer to Vertebrate Animals checklist.

Biohazards. Reviewers will assess whether materials or procedures proposed are potentially hazardous to research personnel and/or the environment, and if needed, determine whether adequate protection is proposed.

Resubmission. When reviewing a Resubmission application (formerly called an amended application), the committee will evaluate the application as now presented, taking into consideration the responses to comments from the previous scientific review group and changes made to the project.

Renewal. When reviewing a Renewal application (formerly called a competing continuation application), the committee will consider the progress made in the last funding period.

Revision. When reviewing a Revision application (formerly called a competing supplement application), the committee will consider the appropriateness of the proposed expansion of the scope of the project. If the Revision application relates to a specific line of investigation presented in the original application that was not recommended for approval by the committee, then the committee will consider whether the responses to comments from the previous scientific review group are adequate and whether substantial changes are clearly evident. For additional information, see http://grants.nih.gov/grants/peer/guidelines_general/Revision_Applications.pdf.

Budget and Period Support. Reviewers will consider whether the budget and the requested period of support are fully justified and reasonable in relation to the proposed research. For more details, please see Budget Information.

Select Agents. Reviewers will assess the information provided in this section of the application, including 1) the Select Agent(s) to be used in the proposed research, 2) the registration status of all entities where Select Agent(s) will be used, 3) the procedures that will be used to monitor possession use and transfer of Select Agent(s), and 4) plans for appropriate biosafety, biocontainment, and security of the Select Agent(s). For more details, please see Select Agents.

Applications from Foreign Organizations. Reviewers will assess whether the project presents special opportunities for furthering research programs through the use of unusual talent, resources, populations, or environmental conditions that exist in other countries and either are not readily available in the United States or augment existing U.S. resources. Reviewers do not need to comment on the foreign component of domestic applications in this review consideration. Comments should be included in the Approach review criterion as applicable.

Resource Sharing Plans. Reviewers will comment on whether the following Resource Sharing Plans, or the rationale for not sharing the following types of resources, are reasonable:

Data Sharing Plan. Applications requesting more than $500,000 direct costs in any year of the proposed research are expected to include a data sharing plan in their application. Certain Program Announcements may request a data sharing plan for all applications regardless of the amount of direct costs. Assess the reasonableness of the data sharing plan or the rationale for not sharing research data. (http://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm)

Sharing Model Organisms. All NIH grant applications are expected to include a description of a specific plan for sharing and distributing unique model organism research resources generated using NIH funding or state why such sharing is restricted or not possible. Unlike the NIH Data Sharing Policy, the submission of a model organism sharing plan is NOT subject to a cost threshold of $500,000 or more in direct costs in any one year, and is expected to be included in all applications where the development of model organisms is anticipated. (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-04-042.html)

Genome Wide Association Studies. Applications and proposals that include GWAS, regardless of the requested costs, are expected to include as part of the Research Plan either a plan for submission of GWAS data to the NIH designated data repository or an appropriate explanation for why submission to the repository will not be possible. (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-08-013.html)

Technology Transfer. Reviewers will assess whether the proposed plan to integrate technology transfer from the partnering organizations is adequate

Additional Comments to the Applicant. Reviewers may provide guidance to the applicant or recommend against resubmission without fundamental revision.

[Return to Review Guidelines Page]