REVIEW AND APPROVAL OF OPERATIONAL GRANTS Background These procedures were developed after extensive consideration of-. (1) the philosophy and purposes of Title IX; (2) the initial experience in reviewing the planning grant applications awarded under Section 903; (3) consideration of the first o erational grant proposals, includin site p 9 visits to the regions involving members of the National Advisory Council on Regional Medical Programs and the Regional Medical Programs Review Committee; (4) preliminary discussion of .the issues involved in the review of operational applications by the National Advisory Council on Regional Medical Programs at its November 1966 meeting; and (5) extensive discussion with both the Review Committee and the National Advisory Council concerning the effectiveness of these procedures during the actual review of the first operational.applications. As a result of these considerations, the resulting review and approval process is to the greatest@ possible extent keyed to the anticipated nature of operational grant requests and to the policy issues inherent in the Regional Medical Programs concept. Characteristics of Operational Grants The review process recognizes the following characteristics of Regional Medical Program grants: (1) complexity of the proposals with many discrete but interrelated activities involving different medical fields-, 2. (2) .;i,l variety of grant proposals resulting from encourage- ment of initiative and determination at the regional level under the flexible parameters provided in the law, regulations, and guidelines; (3) the many different attributes of the overall operational proposal which are appropriate concerns of the review process, including factors affecting the organizatio n and delivery of health services the degree of effective cooperation and commitment of the major medical resources and extending to the evaluation and the merit of highly technical medical activities in the fields of heart disease* cancer, stroke, and related diseases-; (4) the relatio ship of the proposal @9 tho r@ppgnsibiltties of man other qpmpppqn ts of the Public Health Service and other Federal programs requiring a review process that allows adequate opportunity for input from these other sources; (5) the characteristics of thes e.initial proposals as the first steps in the more complete development of the regional medical program, guided by a continuing planning process. Objectives of Review Process The objectives sought in the development of this review process are based on a careful assessment of the goals of the Regional Medical Programs and how the achievement of those goals can be most effectively furthered by the process used in making decisions on the award of grant funds. Consideration of these basic policy issues led to delineation of the following objectives of the review process: 3 1. The operational grant application iiiii.-.t be viewed as a totality rather than as a collection of discrete and separate projects. 2. The- decision-making process for the review and approval of operational grants must be developed in a way that stimulates and preserves the essential goal setting, priority determination$ decision making and evaluation at the regional level. 3. The review process must be concerned with the probability of effective implementation of the proposed activities in addition to the inherent technical merit of the specif ic proposals. 4. The review process must assure a basic level of quality and feasibility of the individual activities that will make an investment of grant funds worthwhile. 5. The review process must-have sufficient flexibility to cope'with the variety of operational proposals submitted, allowing for the tailoring of the review to the needs of the particular proposal. 6. The review process should view a regional medical program as a continuing activity, not a discrete project with time limits; therefore, the review process will have continuity during the grant activity and will have the opportunity to judge the development of Regional Medical Programs on the basis of results and evaluation of progress, in addition to the evaluation of the probable effectiveness of initial proposals. 4 riteria la dical Program grant The basic,criter" for the review of Regional Me requests are set forth in the regulations as follow6: "54.406 Award. Upon recommendation of the National Advisory Council on Regional Medical Programs, and within the limits of available funds, the Surgeon General shall award a grant to those applicants whose approved programs will in his judgment best promote the purposes of Title IX4 In awarding grants, the Surgeon General shall take into consideration, among other relevant factors the following: "(a) Generally, the extent to whichthe proposed program will carry out, through regional cooperation, the purposes of Title IX., within a geographic area. "(b) The capacity of the institutions or agencies within the program, individually and collectively, for research, training, and demonstration activities with respect to Title IX. "(c) The extent to which the applicant or the participants in the program plan to coordinate or have coordinated the regional medical program with other activities supported pursuant to the authority contained in the Public Health Service Act and other Acts of Congressincluding those relating replanning and use of facilities, personnel, equipment, and training of manpower. "(d) The population to be served by the regional medical program and relationships to adjacent or other regional medical programs. "(e) The extent to which all-the health resources of the region have been taken into consideration in the planning and/or establishment of the program. 11(f) The extent to which the participating institutions will utilize existing resources and will continue to seek additional nonfederal resources for carrying out the objectives of the regional medical program. "(g) The geographic distribution of grants throughout the Nation." 5 it was determined In utilizing these criteria in the review process, that the sequence of consideration of the various attributes of the ropbsal would be important if the objectives of the review process listed p above were to be achieved. The review process, therefore, must focus on three general characteristics of the total proposal which separately and yet collectively determine its nature as a comprehensive and potentially effective regional medical program: (1) The first focus must be on those elements of the proposal which identify it as truly representing the e t of a regional medical program. Our review groups have determined that it is not fruitful to consider specific aspects of the proposal unless this first essential determination concerning the core of the program is positive. In making this determination considerations include such questions as: "is there a unifying conceptual strategy which will be the basis for initial priorities of action, evaluation, and future decision making?" "Is there an administrative and coordinating mechanism invol@ing the health resources of the regions which can make effective decisions, relate those decisions to regional needs, and stimulate the essential cooperative effort among the major health interests?" "Will the key leadership of the overall regional medical program provide the necessary guidance and coordination for the development of the program?" "What is the relationship of the planning already undertaken and the ongoing planning process to the iniitial operational proposal?" 6 (2) After having made a positive determination'about this core activity, the next step widens the focus to include both the nature and the effectiveness of the proposed cooperative arrangements. In evaluating the effectiveness of these arrangements atten tion is given to the degree of involvement and commitment of the major health resources, the role of the Regional Advisory Group, and the effectiveness of the proposed activities in strengthening cooperation. Only after-the determination has been made that the proposal reflects a regional medical program concept -and that it will stimulate and strengthen cooperative efforts will a more detailed evaluation of t 6 specific operational activities be made. (3) If both of the two previous evaluations are favorable, the operational activities can then be reviewed, individually and collectively. Each activity is judged for its own intrinsic'merit, for its contribution to the cooperative arrangements, and for the degree to which it includes the core concept of the Regional Medical Programs. It should'also fit' as an integral part of the total'operational activities, and contribute to the overall objectives of the Regional Medical Programs. Review Procedures Attached is a chart which describes the various steps in the review process which will be applied- to initial operational grant proposals from each region, The first four operational grant proposals were subject to the various steps of this process. Those steps were not carried out in .7 ce provided in this chart since the first precisely the order and sequel' ituation fPr the development of this four applications were used as a test S . rience will lead operational procedure. It is also likely that further expe The following comments to appropriate modification of these procedures. may help to explain this review process, which has been agreed to by the Regi onal Medical Programs Review Committee and the National Advisory Council on Regional Xedical Programs. The complexity ofIthese grant requests and the steps in the review process which seems appropriate for their rev ew will- require as much as six months for the completion of the total review process in most cases, (i) Initial Consideration by Review Committee--The first steps of the review process involve preparation for the site visit which will be conducted for each operational grant application. The first consideration of the application by the Review Committee will be for the purposes of, providing information and comments for the guidance of the site vis3t team, utilizing'staff analyses of the planning grant experience, considerations of gross technical validity, policy issues raised by the particular application, and initial input on relationships to other,Federal programs. (2) Site Visit--Initial experience has indicated that a site visit is essential for the assessment of the extent-to which the regional medical program has developed an overall guiding concept for the regional medical program which has been the base of assignment of priorities of action. It also provides the opportunity to assess the probable effectiveness of cooperative arrangements and degree of commitment of the many elements which will be essential to the success of a regional medical program., As the discussion above points but, it is necessary to reach favorable conclusion on these aspects of the regional medical program before it is justifiable to,begin the major investment of the time of the Division staff,, technical reviewers in other parts of the PHS,-technical consultants, and the DRMP review groups, which is necessary for the assessment of the various components of the application. The site visit is not a substitute for the investment of this effort but provides the opportunity to evaluate the cooperative framework of the regional medical program and the overall probability of the success of the proposed program. (3) Intensive Analysis and Technical Reviews--If the site visit re port justifies the investment of additional effort in the review of the application, the review process proceeds with the undertaking of an intensive staff analysis of the application which is the basis for obtaining specific comments from other components of the PHS and other Federal health agencies with related programs, detailed comments from the various components of the DRMP staff, technical site visits on specific projects within the overall application when considered necessary, and the assimi- lation of additional information from the applicant as a result of the site visit. The technical review of specific projects should not be simply from the standpoint of national merit criteria but should identify specific problems with any project which might prevent that project from making a 9 meaningful contribution to the objectives of the regional medical program. Technical reviews also consider the justification for the particular project budget as presented* This aspect of the review process als 0 presents the opportunity to consider possible overlaps and duplications with other PHS programs which can be a factor in determining how much support should be provided for the particular activity from the regional medical program grant. The opportunity to raise these questions is not limited to DRMP staff initiative since copies of all applications are distributed to all bureaus of the PHS at the time of receipt and representation from all buroauo to invited to all I)W review meetings. (4) Second Review by Review Committee and Recommendation for Action-@ The Review Committee considers o_ll of the information available concerning the application. In addition to the application itself and the site visit report, a summary of all available information is presented to the Con'mittee in a staff presentation. The Review Committee then makes its recommendation concerning the application, Because of the complex nature of the applications,' the Review Committee can divide its recommendation into several parts relating to different parts of the application. If there is an overall favorable recommendation, the Review Committee recommends n overall grant amount based on a discussion of the specifics of the application. This amount takes into consideration problems raised by technical reviewers, overlap with other programs, feasibility of the proposals, and,other relevant considerations raised by the review process. While the overall 10 of the specific components of the amount recommended is based an discussion total application, it does not in most cases involve a specific approval ion of individual projects within the application, or reject although the attitude of the review group toward a particular project may be a major determinant in the calculation of the overall amount recommended. (5) Review by National Advisory Council on Regional Medical Programs-- The National Advisory Council then considers the Review Committee recom- mendations. It has available to it the full array of material presented to the Review Committee and a staff summary of that material. Further information obtained by the staff on the instructions of the Review Committee may also be presented. The National Advisory Council then makes its required legal recommendation concerning the application. The Advisory Council also recommends an overall grant amount. The Council may delegate to the staff the authority to negotiate the final grant amount- within set limits. (6) Meeting with applicant--Following the National Advisory Council meeting, the staff of the Division meets with representatives of the applicant and presents to the applicant the recommendation and comments of the Council. If that recommendation is favorable, the staff also presents to the applicant the recommended overall budget ceiling for the grant along with a summation of all the comments derived'from the review process concerning particular activities contained within the application, including negative comments about specific projects and the budget levels proposed for specific projects. The review process does not disapprove spe cific projects even if there may have been considerable doubts raised by the review process except where the project is infeasible, outside the scope of a regional medical program, raises serious overlap questions, or is technically useless. (7) Submission of Revised Proposal--On the basis of this meeting, the applicant submits a revised proposal within the recommended overall budget ceiling, utilizing in the revision the comments and criticisms resulting from the review process. This step of the process requires the applicant to reconsider their priorities and to assume the basic responsibility,for making the final decisions as to the constituent elements of their operational activity after having the benefit of technical comments and advice, The applicant may choose to undertake an activity even if serious doubts about the activity were raised by the review process with the understanding that such an activity will be followed with interpst by the review groups and will be judged in the future on the basis of results. (8) Final Award Decision--Fo.1lowing staff review of the revised proposal, the final award decision is made resulting in the award of a grant. At this stage some final negotiations with the applicant may take place. February 1967 Division of Regional Medical Programs February 1967 FLOW CHART OPERATIONAL GRANT REVIEW AliD APrROVAL PROCESS Operational Grant Application Received Sta ff Review Committee Members Review Committee Meeting ftiitial Staff Information re for Guidance for Site Visit Team n. Planning grant experience Info@ation & Comment b.. Gross technical validity c. Policy issues d. Relationship to other Federal programs review Committee Cuidance Site Visit Judgments re: (Two days) 1. Concept of REP 2. Cooperative Arrangements 3. Relationship of projects, one to another and to the total 4. Approximate magnitude of support warranted Site 5. Quality of projects where (Prepared @nd day by site team) Visit appropriate Report 2 In addition to application and Review Co@i@tee Meeting Actions:' site visit report 1. Additional information from for I.' Recommendations applicant Consideration & Action a. Approval b. Approval with conditions 2. Expert review re quality of project C. Deferral from outside DRNP, where indicated d. Return for revision including comments from other com- e.' Disapproval ponents of the PHS; may have neces- 2. Instructions to Staff sitated technical site visit on 3. Recommendation of an overall specific project(s) grant amount based on discussion 3. Further Staff information of specifics of the application 4. Discussion by site visi.tor(s) of additional information obtained subsequent to site visit. In addition to above: National Council,Meeting Actions: 1. Review Committee recom- for 1. -Recommendations mendations ConsfLderation & Action a. Approval 2. Further Staff information b. Approval with conditions per Committee instructions c. Deferral d. -Return for revision e. Disapproval 2. lpstrivetions to Staff 3. Recommendation of an overall grant amount Provided to Applicant: Applicant action: 1. Recommendation and comments of Meeting between DRMP Submission of revised proposal Council; if overall approval.-- Staff and Applicant within recommended overall budget proceed to 2. Representatives ceiling utilizing the comments and 2. Recommend overall budget ceiling criticism resulting from the for grant review process. 3. Summation of all comments derived from the review process about particular activities staff review of revised proposal rd Decision Action: a. Award of Grant or b. Further negotiation with applicant ISSL'ES IN REVIEW OF OPERATIONAI, APPLI, GATIONS 1. Characteristics of first operational proposals A. Many projects contained,.in each cmVlex proposal B. sizable budget requests,, Including large hardware requests C. Commitment of effort 11. Relevant factors for judgment: Regional Medical Program vs Collediton of projects A. Overall leadership and guiding philosophy I.. Is there a unifying conceptual strategy which will be the basis for initial priorities of action, evaluation, and future decision making? 'Are the-,r,6 sufficient feedback loops in the strategy? 2. Is there an administrative mechanism which can: a. make decisions b. relate to regional needs c. stimulate cooperative effort among major health interests 3. Are the key leadership persons identified? Can they work with the major health interests? Do they have experience and skills appropriate for providing leadership to a complex endeavor? 4. Is there involvement and comitmnt of the major health interests a. Medical schools b. Practicing physicians c. Hospitals d. Public health agencies 5. Will the ongoing planning process interact with the first operational steps in the development of a program that meets the broader needs of the entire region? B. Nature and adequacy of cooperative arrangements C. Relationship of projects to each other and to the@ total regional medical program 2 @)uality standards A. Highly specialized nature of some projects B. Variety of projects included in each proposal. C. Regional vs. National standards D. Block grant support E. Degree of detailed justification expected F. Type of evaluation necessary for final action C. Emphasis on grantees own evaluation mechanisms as quality uplifting factor at regional level IV. Criteria for judging appropriateness of support A. Scope and limitations of Regional Medical Programs legislative authority B. Availability of other sources of support C. Priority on innovative and leverage effects V. Criteria for judging level of support A. Geographic distribution - Should consideration of availability of funds for later proposals be a part of decision on amount awarded to first applicants B. Partial or phased sup-port as mechanism for: 1. Allowing fuller development of plans before proceeding to fuller implementation 1 2. Permitting better decisions on distribution of funds 3. Early review of progress C. Need to support "critical mass" of activity which will have a sufficient impact to permit evaluation of results D. Support of costly activities as national or interregional resources when justified by the involvement of unique capabilities in a specific Regional Medical Program E. Extent of need for support of operational activities as necessary for further development, extension, and solidification of regional cooperative arrangements 3 71. Ikngth of commitment A. Degree of emphasis to be placed on self-limiting nature of projects B. Need for long range commitment for "core" activities which are essential investment for conduct of specific projects VI!, Relationship of operational proposals to ongoing planning activities A. Need for documentation of relationship B. Extent of prior planning and its relationship to proposed operations and continued planning C. Extent to which needs of periphery of the region need to be documented as basis for undertaking operational activities VIII. Need to spell out relationship with adjacent regions IX. Adequacy of administrative arrangements, including fiscal accountability of grantee TENTATIVE OUTLINE OF OPERATIONAL GRANT REVIEW AND APPROVAL PROCESS Operational Grant Application Received Staff tReview Committpe Members 1. Initial Staff Information re a. Planning grant experience Review Committee Meeting b. Gross technical validity for Guidance for Site Visit Team c. Policy issues Informatioia Ccmme-ut d. Relationship to other Federal 'programs 2. Review Committee Information & Comments Judgments re: Review Committee Guidance 1. Concept of PMP Site Visit 2. Cooperative Arrangements (Two days) J. 3. Relationship o' projects, one to another and to the total 4. Approximate ma&nitude of support warranted t 5. Quality of projects where Site appropriate (Prepared 2nd day by site team) Visit Report 2 In addition to application and :> Review Committee Meeting Actions: site visit report: for 1. Recomnendations 1. Additional information Consideration & Action a. Approval from applicant 2. Expert review re quality b. Approval with condi- of proje6t from outside tions DRMP, where indicated; c. Deferral may have necessitated d. Return for revision technical site visit on e. Disa,,Dproval specific project(s) 2. Instructions to Staff 3. Further Staff information 4. Site visitors In addition to above: Actions: 1. Review Committee recommenda- National Co-@-@ncil Meetin 1. Recommendations tions a. Appruval Consideration & Action 2. Further Staff information b. Approval with condi- per Committee instructions tions c. Deferral d. Return for revision e.. Disapproval 2. Instructions to Staff January 1967 (Rerun April 17, 1967)