FOR RELEASE TUESDAY, A.14. JI%NUARY 17, 1967 "PROGRAM EVALUATION"* *Speech by: Vernon E. Wilson, H.D.p Dean, University of Missouri School of Medicine, Columbia, Missouri, and Coordinator Regional Medical Program of !4iscouri Region. Delivered on: January 16, 1967 (3 p.m.) Delivered at: Conference on Regional Medical Programs at Washington Hilton Hotel Washington, D.C. January 15-17, 1967 "PROGRAM EVALUATION" Vernon E. Wilson, M.D. Dean and'Director Tiniversity of Missouri Medical Center Columbia, Missouri The dilemma of a dean from the day of his appointment is to know when to speak out and when to remain silent. Speaking requires at least an acknowledged topic and at.best a brief, flavorful and meaty content., In pursing.the somewhat evanescent title assigned for this topic - which evolved from "Program,rl to "Program and Evaluation," to "Program Evaluation," I must confess that the merit of silence loomed ever more attrac't@". Since detailed discussion of technical evaluation procedures would not be appropriate under our limits of time, let us compromise and discuss some well known principles .o,f - program and, for the health field, some relatively unused principles of.evaluation. We will examine both in the light of opportunities presented by the Regional Medical Program. The challenge.to Regional Medical Prpgrams, as I see it, is to demonstrate that this new -endeavor established primarily in behalf of heart, stroke cancer, and related.diseases, is more than a static assemblage of existing resources.. @ This in itself is a basis for very careful thought. Most of the principles and programs which can be considered in the field 6f health and health care have been.studied by one or another of the existing- governmental, academic, prof,e.ssional,.or voluntary groups. Thus, at the outset it seems apparent that the aim of the Regional Medical Program must be one of synthesis, an effort to.combine these various factors into a whole which will be greater than the sum of,the.parts. We have already.heard that,the appearance of the Regional Medical Program through federal legislation was a direct result of growing public and professional unrest centered around the slow rate at which new knowledge was being put to use. This concern is,not unique.to the health field but it is new as a major emphasis among the concerns of the health care professions. The agricultural and-engineering,experiment stations, long an integral part of the land-grant colleges, re resent one attempt to deal with this problem. The p Engineers already have a term for it. They label this activity the "transfer of technologyit' It would appear then that the special mission of Regional Medical Programs is primarily one of research in the distribution of health care with the focus placed firmly upon the:patient's needs, rather than upon those of the,institution,or the health .professions.' -2- Until the early part of this century the healing arts possessed a dismally small amount of information; consequently the need was primarily for basic medical knowledge. With the momentum now established in basic research we can now give increased emphasis to indirect factors, such as population size, number of related organizations and groups, increased capabilities in communication facilities, and an ever accelerating rate of obsolesence of knowledge. The magnitude of recent Congressional appropriations indicates the need for immediate action. Additional and similar legislation is under serious consideration. The comprehensive health planning act provides a logical outlet for knowledge developed under Regional Medical Programs. Thus research being done in the more limited field of Regional Medical Programs can be of value throughout the total health care field. Because of the large amount of time and money to be expended, realistic evaluation of the results is mandatory. Unfortunately, we are hampered by a lack of effective measurement tools. We must start by using available techniques, while admitting their inadequacies. It is essential that collaborative research in system design for the distribution of health care be initiated in concern with those academic disciplines who have a long tradition in simulation, systems research, and - communication research. Existing resources for use in the design of such systems are impressive indeed. If one looks at the great array of governmental health agencies,. academic institutions, voluntary and professional groups, as well As supportive organizations like welfare agencies, community attion groups and others, it readily becomes apparent that the major problem is not that of creating resources which could appropriately handle the problem but rat er a coordination of those resources into an effective unit. Although to some the comparison may be a bit unpalatable, 1 submit that this is a market and distribution process and should be handled as such. An approach of this kind does not deny the essential, nature of professional and academic contributions; it will require a formal and scientific search for an appropriate relationship between all academicians and professionals whose skills can be helpful. Concurrently, the integrity of the academic and research community must be preserved, both as an internal system and as a part of society at large. Thus the analogy of marketing is in all probability N much more than an analogy. It may prove to be an actual pattern which will -3- provide us with illustrations and some basic principles for fruitful pursuit of the tasks ahead. - : in this special field may I The Distribution-Process.@- As a layman .$ . that-the production and distribution offer the oversimplified explanation process amounts to a coordination of many disciplines, assembled for the contribution which each can make to a single goal. While such grouping of resources, particularly-in,the research_process,.suggests the antithesis of the traditional departmental organization, the,.concept.is not unfamiliar to academic institutions. It is exemplified,frequently in institutes on University campuses, in land grant experiment stations, and research centers. These patterns allow many-di,sciplines.to proceed in a,,systematic fashion in searching for new information and combining,that information into an ord@rly,whole Takin the marketing analogy one step further, the rational distribution 9 process would be :simulated and developed as follows: The first step is the establishment of nded,,,,either:recognized or unrecognized. :The next step, after the need.is, determined, is to define it and to create recognition of that specific need in both,the consumer and producer. Here@ we have h direct. paral.lel, w.i,th the opportunities open to Regional-Medicil Programs. Third. @H'aviifg. identif i:ed a, spp-cif 3.c need-. or, needs, it is necessary to undertake basic and'applied.-research-in materials.' resources and their synthesis. The medic-a:l:profession.has @xp@nded proportionately small amounts of its dwn-'O-'netgies in -this endeavor and at. the same.time,h,a,s frequently poorly ut-il-i.:t.6d the contributions which,.c,o,uld be, made by other disciplines. Fotirth.,@ Having Completed the basic research and formulated working models, the next step is the production and delivery of materials and services which may come fro'm'4a variety of places. the ..analogy the patient may move to th@ resources 'or the re6our-ces may be brought to the..patient., but finally the delivery proc7e@s:-requir.es that-the,.end product ofhealth care be synthesized- in, a coordinated and personalized manner for the benefit of the consumer. Market-Identification. 'If we @considet health care.,in the.light of the patient's need, .recognized ot@unrecognized,@ the first painful but necessary. step will be a shift in emphasis..,Much basic research has been sponsored -4- upon the assumption that improvement of tie professions and institutions will automatically benefit patients. However, it may be that the goals of the patient and those of the profession are not always the same. To accomplish our task we must now direct extensive study toward the patient and his needs within the context of his normal pattern of living. Professional action has classically been one of response after the patient requests and is given access to the formal health care system. We must now accept responsibility for-health care of the public as a dynamic, intimate part of daily performance. Identification of needs for concentrated research endeavors will require the development of end points or goals against which the effect of e performance can be measured. Unfortunately, at present, change in qualitativ such end points are few and largely unproven. 'Most of the measurement systems currently used in the health professions alitative in nature. We can measure quite are quantitative rather than qu adequately deaths,, morbidity, numbers of personnel, and similar items, s by which we can test the impact of health care.upon but we have few mean the daily performance of a given individual. Thus, our first requirement is for a measurement system which can assess the ability of the individual to perform as a useful member of society and his own attitude toward that .performance. Also required will be a,measurement of tie social or peer group's estimate ot the value of the individual's contribution to the group and their attitude toward that contribution. No single factor pan be used as the sole parameter, but when assembled as a pattern these factors should provide at least the first steps in a qualitative measurement of health care. Since diagnosis is also a part of market,definition and since diagnos s opens communic ations betw.een professional individuals, early detection of disease would appear to be a logical first research effort for improvement in the distribution of health care. -Such research avoids the necessity of premature decisions having to do with delivery of health care and would allow a "tooling up" of the communications system under reduced emotional tension.- Much diagnostic support can be provided to individual practitioners with a minimal change in their present practice patterns. -5- Status of the patient needs study. Interaction between individuals is heavily influenced by the status, stated or felt, of each person. We are .preparing major changes in the status of the patient in the health care system. This calls for a "shorthand" method interwoven in the system itself to assess status, and change in status, particularly of the patient. An interesting correlation exists between the way we use the time of others and our estimate of their importance. Consequently, accurate determination of our expenditure of the patient's time through the design of health services is accessible, measureable, and potentially valuable. Another little us ed field of knowledge is that developed in advertising research. Significant portions of established kn owledge About health is not utilized even by those best acquainted with it. Advertising research has a rich body,of basic knowledge and techniques dealing with facilitators, or why people choose one service or product as opposed to another. These tools and techniques used so successfully in advertising could be adapted and should be useful in broadening public education and personal responsibility in health care. Turning to the third item in our analogy research in materials and resources, we should comment first on basic research which has a long university tradition and is the foundation upon which applied research is conducted. Basic research in almost all academic disciplines will make important contributions to health care.' High on the list should be research in. synthesis of systems, including model building. In our past, testing through models has had.little systematic and comprehensive attention. It could produce large savings in time, -as well as funds, but will require the talents of a variety of existing disciplines, the engineers for exam le, who until recently were seldom formally invited into the health p research conversation. An interesting facet of the dilemma related to manpower shows in the fact that although we.lare faced with a tremendous shortage of health personnel and a low level of national unemployment, we as a health care'group have largely ignored one of our greatest potentials - the patient himself. He is usually the most involved, often the better educated and certainly the most highly motivated party in the interchange, yet we have assigned him the most passive role. Patients, I submit, may not be so helpless as some of our practices would seem to imply. Our friends in sociology should be able to help us here. -6- In the fourth and final phase of our analogy, we will face a variety of problems in the delivery of health care. These include implementation of research and development in distribution. All pIatients should have.access to the best source of care regardless of geography, financial resources., or special interests of particular Professional groups. New patterns are p required. The relationship between centers of excellence and the population which they would serve will need'to be defined. Most organizations which support health care use politically determined boundaries, i. e. , the 'ciIty, county or state. The probability 'of gaining coordinated support from all' interested organizations for the assistance of@a single and specific' individual will be'enhanced 'by la maximum overlap in geographical areas of designed responsibility. A second problem to be considered deals with control. Should such; distribution systems be totally'under'the, control of the health professions? If not, how much of the process should be conducted in cooperation with other interested groups? When should;control be turned to them? A third problem concerns the obsolescent mind, both as it relates to the medical professio@n@itseif and to the public at large. It is clear that planned, continuing education for the profession and the 'ublic is p necessary. A searching look at potential in@i6grati'o-,i of such education with the care process seems called for. back mechanisms must be 8is of cause 'or,. at established for a progressive an@ly and effect least, correlation between continuing education and change. A successful distribution system will itself require an integrated information service. Info'r'matio'n should be'derivo-d from the home, from the avenue of access toIthe health care system, the local hospital, and the large medical center. It will require the development of: conmon'identifii2ation systems and vocabularies. Many of us hope that in the very near future"the .1 social security number will be issued at the time of birth, or,entry'into the country @and will provide such identification. The d information propose system should be designed to utilize, assist 'and refine present systems, not compete with them. The decision for diagnosis- and treatment of the'patient' will take into account his desires which, among other things, relate to the dis,tafi'ce from' 'd confidence in the recommended health care and the patiefit's knowledge of an resource. Other considerations are the ad'equacy'of the health care resources,@,- -7- the cost to the patient and the involved agencies, and the maximum benefit from the care process which includes such by-products as education, research, and economic impact upon the community at large. Finally as we have already heard,, no matter how one may describe a Medical Region, it must interact with other regions. Mechanisms must be developed which will minimize the mechanical problems of inter-regional relationships and permit us to focus upon t e patient. THE EXAMPLE. With no claims to assured success, the Missouri Regional Program has attempted to face these challenges in the planning process. Projects will arise from community groups and be funneled through a refinement process. This should encourage maximum motivation and participation at the grassroots level. A general objective of the program is the development of models of early detection integrated with continuing education. Primary emphasis will be placed on those endeavors which can be quantitatively evaluated and the initial assumption is made that adequate information and communication will provide qualitative improvement. The long range plan provides for qualitative measurement of delivered health care. Only a few projects -are.. proposed for studies of delivery of care. It is our intent simply to be supportive to existing care patterns while setting up the necessary information gathering mechanisms. Under this plan, a request for information by the physician will be met by specific answer to the question, along with additional synoptic background information or bibliographies which should be helpful in his continuing educa tion. Such inquiries will also serve as a guide to the physician's needs. In this manner diagnostic and delivery patterns of health care can quickly be modified in detail when research indicates the desirability of doing so. The data handling facility developed at the University of Missouri: for the purpose of extending the competency of the physician will be integrated with cooperative data handling programs established by hospitals, physician.is offices and state agencies. This integrated system is expected to furnish feed back and monitoring which will make it possible to provide the deIsired information while studying and coordinating the total process in an objective and efficient manner. A University multidiscipline research unit is developing new tools with which to measure achievement. Its staff members have joint appointments with other schools on campus, including N,,@,rsing, Education, Engineering, Journalisin, Business and Public Administration. Libe@r'al Arts and Veterinary Medicine. Presently members of this unit are st-,idviig two different communities in which they will measure efforts toward community health goals, such as rehabilitation of the patient, family reactions and the like. In conclusion, let us review, quite briefly, :;.-m@ goalstworthy of consideration. These goals were picked beca-uce pr.7gre-z-a toward them can be measured. Their evaluation should give us s,,)me insight intd-whether or not we are moving in the direction that may be most effective in meeting the actual needs of patients. 1. The primary goal is to deliver the highest percentage of quality patient care as close to the patient as possible. This is-not'only economical in the total picture but in keeping with the des3.rei§ of most patients. Certainly the latter ass,=ption merits study., 2. Every patient should have equal access t-o any needed national resource. For very special services which are not aviliible in the area, patients can be sent to centers f exc llenc elsewhere, thus eliminating 0 e e th e necessity for duplication of expensive equipment, staff and facilities. 3. Maximum coordination will be sought between the inputs of those; who provide health care directly as well as those involved in@supporting that care,, such as welfare., community resources, environmental control groups, and others. 4. The development of programs to Assist in early and'effective detection of disease will be a primary goal. The information gained can be used to effect changes in delivery of health car'e,@both through personnel, and systems. Early detection is perhap's least threatening to the present health care professions and is among the easiest procedures'to measure quantitatively. It alqn possesses the highest potential for successful qualitative measurements of health care. 5. Post grad-iate education should be "integrated with detection and health care systems. 6. Lay health education will be a vit al part of the regional program. Existing programs .in adult education, extension and voluntary organizations -9- will be utilized so that the -potential recipient of care may be informed as to the role which his physician, the hospital, and the various supporting agencies will play and to the things -which he, the patient, can expect. We need more scientifically designed studies of public attitudes toward health care. 7. Finally, in my view, a crucial goal will be for each of the several regions to take a unique approach to the special needs of their particular areas. Through meetings such as this one, we can share ideas so that a minimum of waste will ensue as we seek to me et our respective responsibilities. New paths, I would remind you, are seldom explored by faint hearts. Perhaps we need to be mindful that, in the development of new systems, one may at times work with less than perfect parts in order to set the system itself in operation. It is possible, even desirable, to have-"proof runsit a practice long utilized by the printing indust-@"y. From less than perfect initial operations, changes and corrections can be made to.improve the final product. As participants in this national program I believe we dare not do less than marshall the best available talents, from whatever quarters, to join in this quest for improved health care. The opportunities are attractive and challenging to say the least.