Preparing MARC Program (T34) Application Review Reports
Applications should be evaluated using the criteria listed below, emphasizing those aspects that seem most important for each application. Organize your report using the headings shown. Consider the following points when preparing your report:
- Base the evaluation on the information obtained from the application, additional materials, and site visit (where applicable). Focus on the scientific and educational merit of the application; avoid issues that are not directly related to the merit of the proposed program.
- Two to three pages should be adequate to address strengths and weaknesses. Make concise, critical comments focusing on the key positive and negative aspects of the application.
- Including extensive descriptive material in the critique distracts from your assessment. Limit the description of the program to that absolutely needed to make evaluative points.
- Avoid unnecessary extreme comments, positive or negative, and be diplomatic.
- Put the strengths and weaknesses in perspective by indicating their relative magnitude: This is a “critical weakness,” versus a “minor problem,” or a “major strength,” etc.
- Do not make direct references to site visits; observations can and should be included, but try not to preface them with “At the site visit…” or “The program director said at the site visit….”
- Avoid using the first person; it is not necessary to preface a comment with “I think that…”
- Avoid redesigning the program; critique what is in place or planned.
- Include an overall summary that explains how you arrived at the recommended score.
DESCRIPTION
The applicant's description (abstract) is automatically included verbatim in the summary statement and no further written description is required. However, Reviewer 1 should be prepared to introduce the application to the review group by briefly describing the program goals, design, content, and administration as well as relevant background information (e.g., new, renewal, resubmission).
CRITIQUE
Written critiques should be prepared by all assigned reviewers and will be included in the summary statement with little or no editing by NIGMS staff. Discussants need not prepare a complete report but should provide a brief summary of the strengths and weaknesses of the application.
PROGRAM EVALUATION CRITERIA
Training Program:
- Will having the program lead to an increase in the institutional outcomes of alumni/alumnae earning Ph.D.s?
- Are the objectives, design, and direction of the proposed research training program appropriate?
- Does the proposed program provide suitable training for the levels of trainees being proposed and the areas of science to be supported by the program?
- Are the quality of advising and proposed skills development activities appropriate for pre-MARC trainees and trainees?
- Are inter- and multi-disciplinary and inter-professional research training opportunities or novel concepts, approaches, methodologies, or technologies appropriately utilized?
- Does the training plan include required rigorous curricula designed to assure that MARC trainees are prepared for highly selective graduate Ph.D. programs, especially NRSA-supported graduate research training programs?
Training Program Director:
- Does the Program Director have the scientific background, expertise, and administrative experience appropriate to direct, manage, coordinate, and administer the proposed research training program?
- Is the time commitment of the Program Director adequate and feasible?
- Is there an effective team of faculty, staff, and administration that assures continuity of the program?
- Will the program leadership be able to give an appropriate level of effort to the program to ensure the success of its objectives?
Research Mentors:
- Are the in-house and external mentors of appropriate caliber as researchers and do they have a record of preparing underrepresented minorities for further research?
- Are there a sufficient number of experienced mentors with appropriate expertise and funding available at the applicant institution or at a partner institution to support a viable program?
- Is the experience of those providing academic/career advice or workshops appropriate?
Past Training Record:
- Are the past records of 1) student development at the institution and 2) of research training by the program (if a renewal application), of the Program Director and designated preceptors/mentors appropriate?
- How successful are former trainees in seeking graduate training in highly selective research training programs?
- Is there evidence of alumni/alumnae completion of Ph.D. programs and progression to further training and research careers?
Institutional Training Environment, Commitment, and Resources:
- Is the quality of the research environment for the proposed research training program appropriate?
- Is the level of institutional commitment appropriate?
- Is the quality of available facilities, curriculum, research and research training support suitable?
- Will the proposed program be an integral component of the applicant institution’s overall research program/mission?
Trainee Recruitment, Advising, and Selection Plan:
- Are there appropriate plans for identifying, advising, and strengthening potential trainees?
- Are the recruiting procedures, trainee selection criteria, trainee selection process, and retention strategies appropriate and well defined?
- Are there advertising plans or other effective strategies to recruit high-quality trainees?
- For competing renewal applications: How successful has the program been in its efforts to develop trainees who go on to a highly selective Ph.D. program?
Evaluation and Tracking Plan:
- Is the evaluation plan adequate and sufficiently detailed to determine changes in institutional outcomes?
- Are there clear plans to track career outcomes of trainees and to determine if the program is successful?
- Does the evaluation plan include a system for tracking participants following program completion, such as publications, grant proposals and awards, and the career trajectory of supported trainees?
- For competing renewal applications: Are there plans to make changes that improve program performance and the assessment of outcomes?
Overall Summary: Briefly summarize the major strengths and weaknesses of the application that lead to your recommended priority score.
ADDITIONAL REVIEW CRITERIA (IF APPLICABLE)
Protection of Human Subjects from Research Risk:
- Are there adequate protections of human subjects from research risk relating to their participation in the proposed research? See the Decision Tree for Protection of Human Subjects from Research Risk.
- Are there adequate plans to include subjects from each gender, all racial and ethnic groups (and subgroups), and children as appropriate for the scientific goals of the research?
Care and Use of Vertebrate Animals in Research:
- Is there appropriate justification for use and adequate plans for care of any vertebrate animals that are to be used in the project?
Biohazards:
- If materials or procedures are proposed that are potentially hazardous to research personnel and/or the environment, are the proposed protections adequate?
ADDITIONAL REVIEW CONSIDERATIONS
Budget:
- Are the proposed budget and the requested period of support reasonable in relation to the proposed research training program and the number of proposed trainees at the requested levels?
NOTE: The priority score should not be affected by the evaluation of the budget.
Training in the Responsible Conduct of Research:
- Based on the appropriateness of topics, format, amount and nature of faculty participation, and the frequency and duration of instruction, is the plan for student training in the responsible conduct of research adequate? See the NIH Guide's Training in the Responsible Conduct of Research.
NOTE: The plan will be discussed after the overall determination of merit, and the review panel’s evaluation of the plan will not be a factor in the determination of the priority score. Plans will be judged as acceptable or unacceptable, and the result will be described in an administrative note on the summary statement. Regardless of the priority score, applications with unacceptable plans will not be funded until the applicant provides a revised, acceptable plan. The relevant NIGMS staff will judge the acceptability of the revised plan.
