Select for print version (PDF file, 395 KB). PDF Help.
By Robin M. Weinick, Ph.D., Agency for Healthcare Research and Quality; Peter W. Shin, M.P.H., Ph.D., George Washington University
Health care organizations are now emphasizing evidence-based medicine, which involves using research findings on the effectiveness of various practices to help make treatment decisions for patients. A parallel practice that is receiving increased attention is using data and the findings from data analysis to inform the policymaking process. The data-driven policy framework presented here involves an explicit statement of priorities and policy questions to be answered by new and existing data and provides general guidance for using data to support the process of developing policy options for the health care safety net.
The Safety Net Monitoring Initiative
A Process for Developing Data-Driven Capabilities to Support Policy Decisions
Who Should Be at the Table for This Process?
Definitions and Priorities
Data
Assessment
Action
Evaluating the Policy
Using Data for Developing Policy Options
References
In September 2003, the Agency for Healthcare Research and Quality (AHRQ) published two data books in cooperation with the Health Resources and Services Administration (HRSA). These books are entitled:
These books provide 118 measures to help policymakers, planners, and analysts monitor the safety net in 90 metropolitan areas and all 1,818 metropolitan and nonmetropolitan counties in 30 States and the District of Columbia. In conjunction with a third volume, Monitoring the Health Care Safety Net—Book III: Tools for Monitoring the Health Care Safety Net, these books form the core of the joint AHRQ-HRSA Safety Net Monitoring Initiative.1-3
To further assist State policymakers in using the information in these books, HRSA has funded a grant through the National Governor's Association Center for Best Practices entitled "Enhancing the Safety Net Through Data-Driven Policy." This intensive technical assistance project is designed to help policymakers in four States develop a series of data-driven recommendations to enhance the strength, structure, and stability of their health care safety nets. As part of the project, interdisciplinary State teams are using the two data books to access new information about their safety net systems, and are using these data tools as the basis for crafting their own policy initiatives to strengthen and sustain the health care safety net. (For more information, go to www.nga.org.)
Site visits to the four States (Arizona, Florida, Oregon, and Virginia) selected for this project were conducted in January and February 2003. This data-driven policy framework focuses on the process that evolved from those visits by which States can begin to develop their capacity for formulating data-driven policy concerning the provision, financing, and monitoring of the safety net.
|
For more information on these data books and the Safety Net Monitoring Initiative, go to www.ahrq.gov/data/safetynet to download the books or request free copies, or call (800) 358-9295 and request publication numbers 03-0025/03-0026. |
Figure 1 (23 KB) displays a four-stage process for developing data-driven capabilities to support policy decisionmaking.
Ideally, this process should involve all the major stakeholders with an interest in the safety net. This includes State agencies that are involved in providing policy direction, program management, and data collection. In addition, including representatives of associations of community health centers, free clinics, and hospitals as well as other providers involved in the safety net would be beneficial as well. Moreover, representatives from county and city health departments, special State and local programs, and organizations that collect health data add tremendous depth of perspective.
To help facilitate discussion, the agency or official responsible for convening the stakeholder organizations also should define the focus and charge of the safety net workgroup. For example, the workgroup may be charged with identifying ways to improve provider participation in the State Medicaid program or identifying data to help describe access problems in the safety net. Specifying the purpose of the work group serves to ensure that appropriate representatives (e.g., chief executive officers [CEOs] or data experts) are present to provide direction for developing viable policy options. In effect, this policy process relies on the workgroup involved to clarify questions about the safety net, share data and information, assess policy options, and describe policy recommendations sufficiently for effective implementation.
All parties involved must begin by articulating a common definition of the safety net, which is usually defined in three primary ways:
These are not mutually exclusive, and States may choose to incorporate multiple aspects of each of these three dimensions in their definition.
Populations may include:
Providers may include:
Funding streams may include:
Failing to begin with a shared definition of the safety net may cause confusion later in the process, as representatives from different organizations may be working under different assumptions.
As the process begins, members of the workgroup should understand each other's respective concerns and priorities regarding the safety net. Developing a shared list of concerns and priorities can shape the overall direction of the project and help to more efficiently target data-based policy efforts. For example, the group may choose to focus its efforts on one or two priority areas, such as measuring capacity for primary care or mental health services for the uninsured, rather than beginning with a general review of all data available that may be relevant to the safety net.
In the initial stages of formulating data-based policy options, begin by identifying what questions or policy problems the work group feels should be addressed. For example, if the group is convened to address the financial fragility or vulnerability of the safety net, they may inquire about the level of Medicaid revenues, uncompensated care pool payments, or local indigent program funds paid to safety net providers.
Figure 2 describes a data matrix that can help evaluate what data are available to support the policy development process. A data matrix serves as a reference tool for everyone working on the project, so that there is a clear understanding of what resources are available for answering key questions about the safety net. It should include all relevant data sources, regardless of where they are housed, and is likely to include data sets from multiple State and local agencies, Federal data collection efforts where relevant, and from non-governmental organizations that collect data (e.g., an association of community health centers).
For each data set that contributes to analyzing the priorities, the following questions should be addressed:
Figure 3 provides a sample data matrix for the data sets used for the two data books from the Safety Net Monitoring Initiative.
Without appropriate measures, determining baseline conditions or evaluating the impact of policy changes can be difficult. With that in mind, it is particularly helpful to understand what specific measures are available from existing data sources that can help track the effect of any actions.
Ideally, measures are:
Examples of measure regarding the safety net include emergency department visits and cost of care per uninsured patient. It may be necessary to use a proxy measure, if a more direct, closely related measure is unavailable. For example, access to primary care can be measured by the volume and increase in emergency department visits for non-emergent care.
Existing data resources may not be adequate to answer all the questions, provide a complete assessment of the current situation, or monitor anticipated changes. Although developing new data may not be a feasible option, it is always helpful to have a clear understanding of the limitations of currently available data and to know about additional or new data that would be helpful. It can be difficult to adequately manage or evaluate programs if data and measures are unavailable, so take the time to identify the limitations of current resources.
In combination with data and measures, an inventory of current and past initiatives focused on the safety net can help provide context for developing new policy options.
Once definitions and priorities have been established and a data matrix has been constructed and shared among the relevant stakeholders, the data can be used to assess specific issues or problems to target policy options to the specific providers, populations, geographic areas, and so on, with the greatest need.
Figure 4 (13 KB) shows the Policy Analysis Framework originally developed by Larry Lewin, Jack Needleman, and David Helms for AHRQ's User Liaison Program.4 It provides general guidelines for using data to analyze the problem with appropriate questions and to assess the available options based on select criteria.
The matrix shown in Figure 2 can help identify data that can be used to answer these questions, as it is designed to answer commonly asked questions about the safety net with available data and to serve as a reference tool for other potential policy efforts.
Specific questions that are recommended for general analyses of the safety net are shown in the Safety Net Assessment Worksheet in the Appendix. Additional questions may be relevant, depending on the definitions and priorities that were selected at the beginning of the data-to-policy process.
Understanding the limitations of current knowledge is crucial for using data wisely to support the policy development process. Be as explicit as possible in outlining the limits of what can be understood with current resources so as to be clear on the "blind spots" in the policy analysis process.
It is often helpful to disseminate the findings from the assessment before or while developing policy options. Stakeholders may have a wide variety of uses for the information, and may also supply creative, viable policy options once they have had a chance to review the results.
|
Go to the chapter, "Presenting Information to Decisionmakers: A Guide for Policy Analysts," by Larry Lewin and Marion Ein Lewin, in Monitoring the Health Care Safety Net—Book III: Tools for Monitoring the Health Care Safety Net. To request free copies of this book, call (800) 358-9295 and request publication number 03-0027. |
The Policy Analysis Framework shown in Figure 4 moves from an analysis of the problem, discussed above, to assessing and selecting policy options. Helpful questions to answer include are listed in Table 1.
A description of the impact that current and previous initiatives have had can be helpful to understanding the likely effects of current options. It is particularly helpful to know about the unintended consequences of these initiatives to minimize such effects from any new policy actions. In addition, consider the evaluation methods for current policy options before they are implemented. Failing to do so may lead to options whose successes and failures cannot be readily measured.
Options and recommendations should be presented in clear terms, with graphic tools to help stakeholders understand the details involved.4 It is crucial to present the results of the data analyses that support the policy options being recommended. This helps garner support by providing a sound basis of evidence for the actions proposed.
While this discussion began with establishing definitions and priorities, ideally the data-to-policy process forms a continual feedback loop. Once a policy option is chosen and implemented, repeated assessment can help measure its impact, establish its strengths and weaknesses, and provide information to support decisions regarding continuation of the initiative or a shift in strategic direction. Similarly, the effects of a policy initiative may change definitions and priorities for future work, beginning the cycle anew.
The Policy Analysis Framework also provides some helpful questions to ask when deciding among the available options as well as the implementation, support, and evaluation of a policy initiative. Some of these are outlined in Table 2.
Figure 5 (11 KB) illustrates how both data and policy priorities influence policy options. Priorities provide a "top-down" perspective—from the big picture of a set of goals and areas of emphasis down to specific policy options. This approach involves a straightforward guideline for identifying relevant policy questions and goals first. The data analysis is regarded as the next step in which collection of new data may be required to support the information needs of the policy process. In contrast, a "bottom-up perspective" relies largely on existing data sources to identify or clarify areas of concern and to provide available evidence for action to contribute to the policy process.
In general, a top-down perspective provides greater flexibility in the scope of the safety net issues the workgroup can address, whereas a bottom-up perspective helps to identify current problems with existing data but may not adequately reflect policy priorities. Although the choice of perspectives in the policy development process may be influenced by stakeholder perceptions, political feasibility of policy options, timing, and available funding, data analysis remains crucial in the policymaking process.
As Figure 5 (11 KB) shows, data analysis can provide either supportive or contradictory evidence for the severity, prevalence, and causes of the problem and the development of reasonable policy options. In addition to the scope and seriousness of the problem, data analysis also can help maximize the desired outcomes of any policy initiative. In general, effective data-driven policy involves both perspectives, one focusing on the listing of priorities and policy questions to be answered by new and existing data and the other relying on a data matrix that can be used as a prepared reference for answering safety net questions as they arise.
1. Billings J, Weinick RM. Monitoring the Health Care Safety Net—Book I: A Data Book for Metropolitan Areas. Rockville, MD: Agency for Healthcare Research and Quality; 2003. AHRQ Publication No. 03-0025.
2. Billings J, Weinick RM. Monitoring the Health Care Safety Net—Book II: A Data Book for States and Counties. Rockville, MD: Agency for Healthcare Research and Quality; 2003. AHRQ Publication No. 03-0026.
3. Billings J, Weinick RM. Monitoring the Health Care Safety Net—Book III: Tools for Monitoring the Health Care Safety Net. Rockville, MD: Agency for Healthcare Research and Quality; 2003. AHRQ Publication No. 03-0027.
4. Lewin L, Lewin ME. Presenting information to decisionmakers: a guide for policy analysts. In: Billings J and Weinick RM, editors. Monitoring the Health Care Safety Net—Book III: Tools for Monitoring the Health Care Safety Net. Rockville, MD: Agency for Healthcare Research and Quality; 2003 Dec. AHRQ Publication No. 03-0027.
AHRQ Publication No. 04-0037
Current as of April 2004
Internet Citation:
Weinick RM, Shin PW. Developing Data-Driven Capabilities to Support Policymaking. Monitoring the Health Care Safety Net. AHRQ Publication No. 04-0037, April 2004. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/data/safetynet/weinick.htm
Return to Monitoring the Safety Net
Data & Surveys
AHRQ Home Page
Department of Health and Human Services