Review Procedures: Scientific Review Group (SRG) Meetings : NIDDK

Review Procedures: Scientific Review Group (SRG) Meetings

  • Review Procedures
  • Minority Opinion
  • Numerical Rating
  • Budget
  • Overlap
  • Foreign Organizations
  • Research Involving Human Subjects
  • Gender and Minorities Inclusion in Clinical Research
  • Research Involving Vertebrate Animals
  • Biohazard
  • Avoiding Conflicts of Interest During SRG Meetings
  • Confidentiality and Communications with Investigators
  • Misconduct

    Also see: NIH OER Page: Peer Review Policies and Procedures

    Review Procedures

    The scientific review group (SRG) evaluates the scientific merit of each grant application according to specific criteria in one of its componant study sections. The principal criteria for the initial review of research project grant applications, based on the Public Health Service (PHS) Scientific Peer Review Regulations, include

    • scientific and technical, or medical, significance and innovation of the goals of the proposed research;
    • appropriateness and adequacy of the experimental approach and methodology proposed to carry out the research;
    • qualifications and research experience of the principal investigator and staff, particularly but not exclusively in the area of the proposed research;
    • availability of resources necessary to conduct the research;
    • the appropriateness of the proposed budget and duration relative to the proposed research; and
    • where an application involves activities that could have an adverse effect on humans, animals, or the environment, the adequacy of the proposed means for protecting against or minimizing such effects.

    The specific criteria will vary with types of applications reviewed, such as the National Research Service Awards (fellowships), Research Career Development Awards, or Small Business Innovation Research grants.

    In addition, for renewal and supplemental applications, preliminary data and/or progress to date must be evaluated. For amended applications, reviewers must evaluate the investigator's response to the previous review and indicate whether the application is improved, the same as, or worse than the previous submission.

    During the meeting, the Chairperson of the SRG, following an agenda prepared by the Scientific Review Administrator, introduces each application, calls upon the individuals assigned by the Scientific Review Administrator to present their written comments, and invites discussion. The assigned discussants are then asked for their comments, and group discussion follows.

    After sufficient discussion has ensued, the Chairperson calls for a priority rating to be assigned to the application if it appears from discussions that the application has significant and substantial scientific and technical merit.

    If it appears that the application is not competitive against the scientific standards of the field, a motion may be made by the review group that the application is not scored (NS). This action is made by majority vote of the review committee. In addition, if there are serious concerns regarding the use of human subjects or animal welfare, a motion may be initiated that the application not be recommended for further consideration.

    If additional information is needed before a review group can make a recommendation, a motion for deferral may be entertained. The review group, by majority vote, may defer an application in order to obtain additional information or for a project site visit.

    [Top]

    Minority Opinion

    If a reviewer moves that the application not be recommended for further consideration and the motion is seconded, the Chair will ask for additional discussion before calling for a vote on the motion. If the motion passes , the application can no longer be considered for funding; if the motion fails , the Chair instructs the reviewers to score the application. If either vote is not unanimous, two or more dissenting members must provide a minority report, which is included in the summary statement; one dissenting member may provide a minority report.

    For any motion of no further consideration, the full action of the committee must be recorded: number of votes for the motion, number of votes against the motion, and number of abstentions. Reviewers are encouraged not to abstain. However, if a reviewer is unable to assess the merit of an application without additional information, as evidenced by his/her prior discussion or recommendation for deferral, that reviewer should abstain from voting on the motion. No minority report is needed when a motion for deferral is voted against by two or more reviewers.

    [Top]

    Numerical Rating

    For each application that the majority of SRG members believe has significant and substantial scientific merit, each person (including dissenting members) records a numerical rating that reflects the individual's opinion of the merit of the application. Numerical ratings range with increments of 0.1 as follows:

    Outstanding 1.0 - 1.5
    Excellent 1.5 - 2.0
    Very Good 2.0 - 2.5
    Good 2.5 - 3.5
    Acceptable* 3.5 - 5.0
    * Has scientific merit and is
    worthy of further consideration.

    Abstaining members do not assign a numerical rating and are not counted in calculating the average of the individual ratings.

    [Top]

    Budget

    The budget recommendation should be based upon careful scrutiny of direct cost of the proposed research for each year of support requested. For all R01 applications with budgets of $250,000 or less, the modular budget format must be used. A detailed budget must be supplied for requests of more than $250,000, and particular attention should be given to the need for all personnel listed in the application and their percent effort in relation to the workscope, the justification for requests in all budget categories, and addition of personnel, equipment, etc., in future years. The SRG can recommend changes in the duration and amount of support if they are scientifically justified.

    [Top]

    Overlap

    Reviewers will identify any apparent scientific or budgetary overlap with active, pending, or planned support. However, potential overlap must not be the reason for altering the budget, thus affecting the priority score. Information regarding potential overlap is included as an Administrative Note in the summary statement and will be resolved by NIH staff in the event of an award.

    [Top]

    Foreign Organizations

    In addition to the regular review criteria, foreign applications are evaluated in terms of special talents, resources (human subjects, animals, diseases, equipment, or technologies), populations, or environmental conditions in the applicant country that are not readily available in the United States or that augment existing U.S. resources. In addition, notice is taken whether similar research is being done in the United States and whether there is a need for additional research in the area of the application. These special review criteria are not applied to applications from domestic institutions that include a significant foreign component.

    [Top]

    Research Involving Human Subjects

    Applicant organizations have the primary responsibility for safeguarding the rights and welfare of individuals who participate as subjects in research activities supported by the NIH. However, the NIH also relies on its SRGs and National Advisory Councils or Boards to evaluate all applications and proposals involving human subjects for compliance with the Department of Health and Human Services human subjects regulations.

    The regulations define "human subject" as a "living individual about whom an investigator obtains (1) data through intervention or interaction with the individual, or (2) identifiable private information." The regulations extend to the use of human organs, tissues, and body fluids from individually identifiable human subjects as well as to graphic, written, or recorded information derived from individually identifiable human subjects. The use of autopsy materials is governed by applicable State and local law and is not directly regulated by the Federal human subject regulations.

    The Department will fund research covered by these regulations only if the institution has filed an assurance with the Office for Human Research Protections (OHRP) and has certified that the research has been approved by an Institutional Review Board (IRB) and is subject to continuing review by the IRB. When research involves only minimal risk and meets certain other conditions, the IRB may waive the requirement for obtaining informed consent. In addition, certain research that poses little or no risk to human subjects is exempt from IRB review and approval. In such cases, however, adherence to ethical standards and pertinent laws is still required. In the past, certification from the IRB was required along with an application. Recently, this policy was changed to allow applications that use human subjects to be submitted without prior IRB approval (see notice). If a grant receives a good score upon review, the principal investigator should begin the process to obtain approval of his or her IRB. No application can be funded without this approval. In addition, the principal investigator and all key personnel must submit a description of education they have completed in the protection of human subjects (see notice).

    The review by the SRG is expected to reflect existing codes adopted by disciplines relevant to the research or the collective standards of the professions represented by the membership. The evaluation by SRG members is to take into consideration the investigator's response to the following points.

      1. Provide a detailed description of the proposed involvement of human subjects in the work previously outlined in the experimental design and methods section. Describe the characteristics of the subject population, including their anticipated number and age range, and summarize the gender and racial/ethnic composition of the population and their health status.
      2. Identify the criteria for inclusion or exclusion of any subpopulation. If gender representation and/or inclusion of minorities and children are not addressed, provide a clear rationale for their exclusion. Explain the rationale for the involvement of special classes of subjects, if any, such as fetuses, pregnant women, children, prisoners, other institutionalized individuals, or others who are likely to be vulnerable, as well as human in vitro fertilization.
      3. Identify the sources of research materials obtained from individually identifiable living human subjects in the form of specimens, records, or data. Indicate whether the material or data will be obtained specifically for research purposes or whether use will be made of existing specimens, records, or data.
      4. Describe plans for the recruitment of subjects and the procedures to be followed, including the circumstances under which consent will be sought and obtained, who will seek it, the nature of the information to be provided to prospective subjects, and the method of documenting consent. State if the IRB has authorized a modification or waiver of the elements of consent or the requirement for documentation of consent. The consent form, which must have IRB approval, should be submitted to the PHS only on request.
      5. Describe any potential risks--physical, psychological, social, legal, or other--and assess their likelihood and seriousness. Where appropriate, describe alternative treatments and procedures that might be advantageous to the subjects.
      6. Describe the procedures for protecting against or minimizing any potential risks, including risks to confidentiality, and assess their likely effectiveness. Where appropriate, discuss provisions for ensuring necessary medical or professional intervention in the event of adverse effects to the subjects. Also, where appropriate, describe the provisions for monitoring the data collected to ensure the safety of subjects.
      7. Discuss why the risks to subjects are reasonable in relation to the anticipated benefits to subjects and to the importance of the knowledge that may reasonably be expected to result.

    If the application designates exemptions from the human subjects regulations, sufficient information must be provided to allow a determination that the designated exemptions are appropriate. If a test article (investigational new drug, device, or biologic) is involved, the test article must be named, and it must be stated whether the 30-day interval has elapsed or has been waived and/or whether use of the test article has been withheld or restricted by the Food and Drug Administration.

    Based on this evaluation, the SRG may

      • score an application without restrictions;
      • score an application, but record comments or expressions of concern to be communicated to the institution and principal investigator;
      • score an application, but limit the scope of the work proposed, impose restrictions, or eliminate objectionable procedures involving human subjects;
      • defer for clarification; or
      • recommend no further consideration be given to the application if the research risks are sufficiently serious and protection against the risks so inadequate as to consider the proposed research unacceptable.

    Any comments or concerns that SRG members may wish to express regarding the adequacy of protections afforded human subjects will be presented as a special note (Human Subjects) in the summary statement. A "concern" is an SRG finding regarding human subjects that requires resolution by program staff prior to award; a "comment" is an SRG observation that will be communicated in a summary statement as a suggestion to the principal investigator. No awards will be made until all expressed concerns about human subjects have been resolved to the satisfaction of the NIH. Specific concerns and policy interpretation requests may be addressed to the Office for Human Research Protections, which is responsible for the administration and interpretation of DHHS policy and regulations for the protection of human subjects of research.

    [Top]

    Gender and Minorities Inclusion in Clinical Research

    The NIH policy requires that, for clinical studies, applicants must include appropriate representation of gender, minorities, and children in study populations, so that research findings can be of benefit to all at risk of the disease, disorder, or condition under study. For the purpose of this policy, clinical research includes all human biomedical and behavioral studies of etiology, epidemiology, prevention (and preventive strategies), diagnosis, or treatment of diseases, disorders, or conditions, including, but not limited to clinical trials. This policy is intended to apply to males and females of all ages. It is recognized that for some research questions, the consideration of gender, age, or minorities may not be germane. If gender, age, or minority representation is not included or is inadequately represented in the research plan, a clear and compelling rationale must be provided in the application.

    The assessment of scientific and technical merit of applications must include an evaluation of the proposed gender, minority, and children composition of the study population and its appropriateness to the scientific objectives of the study. If the representation of gender, age, or minorities in a study design is inadequate to answer the scientific question(s) addressed and justification for the selected study population is inadequate, the reviewer should consider it a scientific weakness or deficiency in the study design, and this must be considered in assigning a numerical rating. For all clinical studies, recommendations on compliance with policy will be noted in a section at the end of the Critique entitled "Gender, Children, and Minority Subjects."

    All biomedical or behavioral research involving human subjects is considered clinical research. Only research projects that meet the NIH definition for phase III clinical trials are coded as clinical trials. Only scored applications are assigned codes. All projects/subprojects should be individually coded and an overall set of codes assigned to the application. Each project will receive gender, children, and minority codes.

    Is this an NIH-defined clinical trial? Y or N

    Examples:
    G1A: Both genders, scientifically acceptable
    M3U: Only non-minorities, scientifically unacceptable

      Gender Codes Minority Codes Children Codes
      First character G M C
      Second character 1=both genders
      2=only women
      3=only men
      4=gender unknown
      1=minority and non-minority
      2=only minority
      3=only non-minority
      4=minority profile unknown
      1=children and adults
      2=only children
      3=only adults
      4= profile unknown
      Third character A=scientifically acceptable
      U=scientifically unacceptable
      A=scientifically acceptable
      U=scientifically unacceptable
      A=scientifically acceptable
      U=scientifically unacceptable

    [Top]

    Research Involving Vertebrate Animals

    Although the recipient institution and investigator bear the major responsibility for the proper care and use of animals, NIH staff, SRGs, and Councils and Boards share this responsibility. Care and use of vertebrate animals in research must conform to applicable law and PHS policy, especially as described in Principles for Use of Animals (see Office of Laboratory Animal Welfare web page for more information). These principles can be summarized as two broad rules:

      • The project should be worthwhile and justified on the basis of anticipated results for the good of society and the contribution to knowledge, and the work should be placed and performed by qualified scientists;
      • Animals should be confined, restrained, transported, cared for, and used in experimental procedures in such a manner as to avoid any unnecessary discomfort, pain, or injury. Special attention must be provided when the proposed research involves dogs, cats, non-human primates, large numbers of animals, or animals that are in short supply or are costly.

    The evaluation by SRG members is to take into consideration the investigator's response to the following five points:

      1. Provide a detailed description of the proposed use of the animals in the work previously outlined in the experimental design and methods section. Identify the species, strains, ages, sex, and numbers of animals to be used in the proposed work.
      2. Justify the use of animals, the choice of species, and the numbers used. If animals are in short supply, costly, or to be used in large numbers, provide additional rationale for their selection and their numbers.
      3. Provide information on the veterinary care of the animals involved.
      4. Describe the procedures for ensuring that discomfort, distress, pain, and injury will be limited to that which is unavoidable in the conduct of scientifically sound research. Describe the use of analgesic, anesthetic, and tranquilizing drugs and/or comfortable restraining devices, where appropriate, to minimize discomfort, distress, pain, and injury.
      5. Describe any euthanasia method to be used and the reasons for its selection. State whether this method is consistent with the recommendations of the Panel on Euthanasia of the American Veterinary Medical Association. If not, present a justification for not following the recommendations.

    Any comments or concerns that SRG members may wish to express regarding the appropriateness of the choice of species and numbers involved, the justification for their use, and the care and maintenance of vertebrate animals used in the project will be discussed in a special note ( Animal Welfare ) in the summary statement. A " concern " is an SRG finding regarding animal care or use that requires resolution by program staff prior to award; a " comment " is an SRG observation that will be communicated in the summary statement as a suggestion to the principal investigator. Questions may be directed to the Office of Laboratory Animal Welfare (OLAW). No award will be made unless the applicant institution has given the NIH OLAW an acceptable assurance of compliance with the PHS policy, and all concerns or questions raised by the SRG have been resolved to the satisfaction of the NIH.

    [Top]

    Biohazard

    The investigator and the sponsoring institution are responsible for protecting the environment and research personnel from hazardous conditions. As with research involving human subjects, reviewers are expected to apply the collective standards of the professions represented within the SRG in identifying potential hazards, such as inappropriate handling of oncogenic viruses, chemical carcinogens, infectious agents, radioactive or explosive materials, or recombinant DNA.

    If an application poses special hazards, these hazards will be identified and any concerns about the adequacy of safety procedures highlighted as a special note (Biohazard, code 44) on the summary statement. No awards will be made until all concerns about hazardous procedures or conditions have been resolved to the satisfaction of the NIH.

    [Top]

    Avoiding Conflicts of Interest During SRG Meetings

    At the beginning of each meeting, the Scientific Review Administrator orients the members by explaining the NIH conflict of interest policy. A member must leave the room when an application submitted by his or her immediate organization is being discussed or where the member, his or her immediate family, or close professional associate(s) has a financial or vested interest in that organization, even if no significant involvement is apparent in the proposal being considered. If the member is available at the principal investigator's institution for discussions; is a provider of services, cell lines, reagents, or other material; or the author of a letter of reference, the member must be absent from the room during the review of that application. Members are also urged to avoid any actions that might give the appearance that a conflict of interest exists, even though he or she believes there may not be an actual conflict of interest. Thus, for example, a member should not participate in the deliberations and actions on any application from a recent student, a recent mentor, or a close personal friend. Judgment must be applied on the question of recency, frequency, and strength of the working relationship between the member and the principal investigator as reflected, for example, in publications. Other examples of conflict might be with an application from a scientist with whom the member has had longstanding differences that could reasonably be viewed as affecting the member's objectivity or of a project that closely duplicates work ongoing in the member's laboratory.

    The term "own organization" includes only the individual institution (i.e., university) in which the member is an employee, consultant, officer, director, or trustee or has a financial interest or with which the member is negotiating or has any arrangement concerning prospective employment. This has been altered from the past policy, which considered that the "own organization" also included all campuses of multicampus universities.

    Each of the Harvard-affiliated organizations is also considered a distinct and separate entity, so that persons from one are not in conflict if reviewing applications from another, provided they have no professional or personal relationships with the other institution. Thus, a member from Massachusetts General Hospital can review an application submitted from Beth Israel Hospital.

    Harvard-affiliated Institutions:

      • Beth Israel Hospital
      • Brigham and Women's Hospital
      • Cambridge Hospital
      • Center for Blood Research
      • Children's Hospital
      • Dana Farber Cancer Institute
      • Frosyth Dental Center
      • Harvard Community Health Plan
      • Massachusetts Eye and Ear Infirmary
      • Massachusetts General Hospital
      • Massachusetts Mental Health Center
      • McLean Hospital
      • Mount Auburn Hospital
      • New England Deaconess Hospital
      • New England Regional Primate Research Center
      • West Roxbury/Brockton Veterans Administration Center

    A reviewer must leave the room during discussion of an application if he or she is a member of, or has a financial interest in, a for-profit organization submitting the application. This includes ownership or stock in or being a consultant for a for-profit organization. A reviewer should also leave the room during discussion of an application if being present would give the appearance of a conflict of interest. An example would be an application from a for-profit organization that provides substantial financial funding to the reviewer's organization or laboratory.

    Prior to the Study Section meeting, each reviewer will receive a certificate of Conflict-of-Interest and Confidentiality and a list of applications that will be reviewed. Reviewers must notify the Scientific Review Administrator of any conflict of interest prior to the meeting and certify that the confidentiality of the review procedures will be maintained.

    At the end of the SRG meeting, the Scientific Review Administrator will obtain written certification from all members that they have not participated in any reviews of applications when their presence would have constituted a real or apparent conflict of interest and that the confidentiality of actions will be maintained. In addition, each study section keeps a log, prepared by the Grants Technical Assistant and maintained in the Study Section Office, of which members left the room because of potential conflicts of interest and for what applications.

    [Top]

    Confidentiality and Communications with Investigators

    All materials pertinent to the applications being reviewed are privileged communications prepared for use only by consultants and NIH staff and should not be shown to or discussed with other individuals. Review group members must not independently solicit opinions or reviews on particular applications or parts thereof from experts outside the pertinent Scientific Review Group. However, members can seek these opinions provided that prior approval from the Scientific Review Administrator is obtained. Members may, however, suggest scientists from whom the Scientific Review Administrator may subsequently obtain advice. Consultants are required to leave all review materials (except for those in the public domain such as journal articles) with the Scientific Review Administrator at the conclusion of the review meeting. Privileged information in grant applications shall not be used to the benefit of the reviewer or shared with anyone.

    Under no circumstance shall consultants advise investigators, their organizations, or anyone else of recommendations or discuss the review proceedings. The investigator may be led into unwise actions on the basis of premature or erroneous information. Such advice also represents a breach of confidentiality for fellow consultants serving on review committees and site visit teams. Such breach could deter qualified consultants from serving on review committees and inhibit those who do serve from engaging in free and full discussion of recommendations.

    Except during site visits, there must be no direct communications between consultants and investigators. Consultants' requests for additional information and telephone inquiries or correspondence from investigators must be directed to the Scientific Review Administrator, who will handle all such communications.

    [Top]

    Misconduct

    "Misconduct" or "misconduct in science" is defined in 42 CFR 50.102 as fabrication, falsification, plagiarism, or other practices that seriously deviate from those that are commonly accepted within the scientific community for proposing, conducting, or reporting research. It does not include honest error or honest differences in interpretation or judgments of data.

    Review of grant/cooperative agreement applications and contract proposals for scientific merit will ordinarily not be delayed by concerns about possible misconduct or by a pending or ongoing inquiry or investigation. To avoid influencing the review process, PHS awarding units generally will not disclose instances of possible misconduct or the status of ongoing investigations. However, if certain instances have received such extensive publicity that the review may be compromised, the agency-level Misconduct Policy Officer (MPO) should consult with the Office of Scientific Integrity (OSI) as to whether the review should be deferred or the reviewers should be informed about the status of activities with regard to the possible misconduct. Findings from completed investigations should be shared with scientific review groups whenever an accurate disclosure of the facts in the case is necessary to the objectivity and thoroughness of the review process.

    During the conduct of the initial review of applications, the SRG may identify instances of suspected or possible misconduct (e.g., suspicions regarding possible plagiarism or questionable data in support of the proposed research). The Scientific Review Administrator (SRA) of the SRG, in consultation with the IRG chairperson and Section Chief, must first determine from the discussions of the SRG if the review may proceed. Generally, what appears to be a relatively minor impropriety, such as the unattributed use of small amounts of textbook material in the background section of the application, would not prevent the SRG from providing a fair review. The general principle is that if the SRG is able to provide an unbiased technical/scientific merit review, unaffected by the suspicions of misconduct, it should do so. Subsequently, the concerns of the SRG will be forwarded by the SRA through the Section Chief to the OSI and the cognizant MPO for resolution.

    In all such cases of suspected misconduct, it is essential that the SRA stress to the reviewers the seriousness of each allegation and the potential harm that may result if confidentiality is not strictly maintained. In addition, it is important for the SRA to assure the consultants that the suspicions identified will be taken seriously and pursued by the PHS. If it is determined that a fair review of an application cannot be carried out by any appropriate initial review group because of the existence of reviewers' concerns about possible misconduct, immediate deferral of the application is the correct course of action.

    In no instance shall the SRA or an SRG member communicate the SRG's concerns to the principal investigator or applicant institution. Instead, immediately following the initial review group meeting, it is the responsibility of the SRA to communicate these concerns to the OSI and the cognizant MPO. As soon as possible thereafter, formal written communication of the concerns and the precise details of the SRG discussion will be forwarded to the OSI by the SRA through the Section Chief and the Cognizant MPO. Any subsequent communications with the principal investigator and/or applicant institution will be done only through the OSI.

    The OSI will attempt to resolve the concerns so that the review may be completed during the next review cycle. If the SRG's concerns are subsequently allayed, review of the application should proceed to conclusion at the subsequent meeting. However, if the concerns remain, and if it is not appropriate or possible for the review to proceed, the application should be deferred while the OSI proceeds with the necessary steps to examine the identified issues.

    It is important that the preparation of the summary statement be carefully monitored by the Section Chief in consultation with the OSI to ensure that only appropriate details of the SRG's concerns are expressed in that document. In addition, the SRA should carefully monitor the priority scoring of the reviewers for that application. If it appears that the scoring has been influenced by the concerns of possible misconduct, the SRA, in consultation with the Section Chief and the OSI, should administratively defer the application for resolution of the identified issues.

    [Top]

Page last updated: November 25, 2008

General inquiries may be addressed to: Office of Communications & Public Liaison
NIDDK, NIH
Building 31. Rm 9A06
31 Center Drive, MSC 2560
Bethesda, MD 20892-2560
USA
For information about NIDDK programs: 301.496.3583

The National Institutes of Health   Department of Health and Human Services   USA.gov is the U.S. government's official web portal to all federal, state, and local government web resources and services.  HONcode Seal - Link to the Health on the Net Foundation