Future Directions for the National Healthcare Quality and Disparities Reports

You Are Here: AHRQ Home > Quality & Patient Safety > Measuring Healthcare Quality > Future Directions for the National Healthcare Quality and Disparities Reports > Improving Presentation of Information

Chapter 6: Improving Presentation of Information

The NHQR and NHDR can be forward-looking documents that not only present historical trend data but also convey a story of the potential for progress and the health benefits to the nation of closing quality and disparity gaps. To serve as catalysts for improvement, the Future Directions committee envisions that the reports will extrapolate rates of change to indicate when gaps between current and recommended care might be closed and will present benchmarks on best-in-class performance. The committee makes suggestions for organizing report content to tell a more complete quality improvement story, realize greater integration between quality improvement and disparities elimination, improve takeaway messages and data displays, and achieve a better match between the AHRQ products and potential audiences.

The NHQR and NHDR provide an enormous amount of data—principally presented in graphs—about how the United States is performing on various measures of health care and how performance has bettered or worsened over time. Although these data are useful, the NHQR and NHDR have potential beyond reporting on historical trends; the reports can also illuminate realistic levels of performance for all to strive toward and provide information on how long it will take to close gaps between current and recommended care at the current pace of improvement.

In this chapter, the committee expands on its vision that future versions of the NHQR and NHDR should tell a more complete story of how to move toward achieving a high-quality, high-value health care system. To make the information presented in the NHQR and NHDR more forward-looking and action-oriented, the committee recommends that AHRQ make greater use of benchmarking and suggests improvements to data displays and the general organization of the NHQR and NHDR. Helping audiences for the NHQR and NHDR better understand gaps in the quality of U.S. health care—whether between actual performance and receiving the recommended standard of care, or between population groups or geographic regions—and better understand the benefits of closing those gaps would provide audiences with stronger evidence and rationales for improving quality.

The chapter begins by reviewing the Future Directions committee's suggestions for how AHRQ's lineup of products could better serve current and expanded audiences. The committee underscores the importance of integrating disparities reduction into quality improvement by enhancing the relationship between the structure of the two national healthcare reports. Finally, suggestions are made on improving data displays and the statistical quality of quality reporting.

Matching Products to Audience Needs

At present, the national healthcare reports and related products are consulted by a variety of stakeholders, many of which have different interest areas (e.g., heart disease, rural health, racial disparities, delivery settings) and different levels of sophistication for data interpretation and analysis. The 2001 IOM report Envisioning the National Health Care Quality Report stated that the NHQR was not to be a "single static report, but rather a collection of annual reports tailored to the needs and interests of particular constituencies" (IOM, 2001, p. 6). The committee believes that AHRQ needs to expand and refine its quality reporting product line to provide products and data that are useful and understandable for a variety of audiences. Therefore, the committee recommends the following:

Recommendation 6: AHRQ should ensure that the content and presentation of its national healthcare reports and related products (print and online) become more actionable, advance recognition of equity as a quality of care issue, and more closely match the needs of users by:
Incorporating priority areas, goals, benchmarks, and links to promising practices.

Redesigning print and online versions of the NHQR and NHDR to be more integrated by recognizing disparities in the NHQR and quality benchmarks in the NHDR.

Taking advantage of online capability to build customized fact sheets and mini-reports.

Enhancing access to the data sources for the reports.

The committee's suggested products, along with their potential audiences, are reviewed in Table 6-1.

Refining the Organization of the NHQR and NHDR

Integrating efforts to improve quality with efforts to eliminate disparities increases opportunities to positively affect change. Presenting the same organizational framework and measures in both reports reinforces users' understanding of the relationship between overall health care quality and the depth of health care disparities. But currently, the two reports are not well linked beyond presenting the same measures.

Changing the Highlights Section of the Reports

The committee proposes that AHRQ present the same Highlights section in both the NHQR and NHDR to underscore the relationship between health care quality and equity. The text of the Highlights section should be developed so that the section can be published as a stand-alone document that could be the subject of dissemination events targeted to relevant stakeholder audiences. The document could:

Spotlight areas with the greatest potential for quality improvement impact and provide detail on what the value of closing quality gaps would be to population health and equity.
Feature progress on priority areas and toward any established national goals.
Discuss evidence-based policies and best practices that may enhance quality improvement or factors that hinder progress as informed by data within the body of the report.
Emphasize takeaway messages directed to different audiences (e.g., policy makers, health care providers, and the public) on what they can do to improve health care quality on prioritized topics and measures.
Include a summary of state performance and the state of disparities.

The committee believes that a summary of state performance should be part of the Highlights section of the reports and would be of interest to legislators and policy makers at the state and national levels. A one- to two-page summary of state performance, perhaps in a scorecard fashion, should be included, and AHRQ could compile this from the information it already provides in the State Snapshots (e.g., ratings from very strong to very weak on overall health care quality, preventive measures, acute care measures, chronic measures, hospital care measures, cancer care measures). Currently, the State Snapshots are not available until several months after the reports have been released; the committee urges AHRQ to include this information in the Highlights section even if the detailed Snapshots are not posted online at the same time. Additionally, a summary on the state of disparities should be included.

Although the Highlights document proposed by the committee would be longer than the current Highlights section, sharing the same Highlights section should streamline AHRQ staff efforts.

Organizing the NHQR and NHDR by the Framework Components

The framework for health care quality and disparities measurement (go to Chapter 3) is both a tool to assess whether balance is achieved in the selection of quality measures and a way to organize the reports. Table 6-2 suggests chapters (or sections) for future iterations of the NHQR and NHDR. To increase parallelism across both national healthcare reports, access to care, a topic currently addressed only in the NHDR, should also be included in future NHQRs. After carefully considering whether efficiency and health systems infrastructure should be featured in both reports, the committee concludes, as discussed in Chapter 3, that efficiency measures of overuse and underuse are of interest for populations included in the NHDR and infrastructure is also applicable to equity concerns. Given the limited nature of measures at this time, however, the same efficiency and infrastructure measures may not always be available to include in both reports.

Incorporating Equity into the NHQR

Based on interviews with current and potential users, the committee finds that, to some extent, the NHQR and NHDR have different audiences. There is one school of thought that improving health care performance overall will ameliorate the problem of disparities in health care; this view tends to neglect the reality that disparities in health care usually persist even as overall performance levels improve. The committee believes that closing equity gaps is one of the most important factors in raising overall health care quality (Chin and Chien, 2006; Clarke et al., 2004). For that reason, the NHQR should incorporate the concept of equity by including an additional section focusing on disparities elimination.

Quantifying the impact of disparities on overall quality performance may be one way to define the connection between health care quality and disparities in the Highlights section of the reports. Furthermore, the commentary on each measure within the NHQR could reflect the degree to which disparities remain or are growing even as quality improves so that conclusions on the state of quality are not misleading. An HIV/AIDS measure reported in the 2008 NHQR provides a concrete example of a situation where the Nation as a whole is performing well but data in the NHQR mask disparities shown in the NHDR (AHRQ, 2009d, p. 65, 2009e, p. 63).

Presenting Data on Priority Populations

The NHDR is required by the 1999 federal law under which it was established to report on "prevailing disparities in health care delivery as it relates to racial factors and socioeconomic factors in priority populations."¹ Priority populations were defined in the authorizing legislation with respect to the agency's full portfolio of activities (research, evaluation, and demonstration projects): low-income groups, minority groups, women, children, the elderly, and individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care. AHRQ's overall activities are also to address inner-city and rural areas. The fourth chapter of the NHDR, "Priority Populations," includes limited supplemental measures specific to each priority population.

AHRQ has presented data on priority populations in the NHDR by offering: (1) summaries of the findings presented earlier in the report on access and on the core measures the NHDR shares with the NHQR (e.g., Tables 4.2 and 4.3 in the 2008 NHDR), and (2) occasional additional measures of particular interest for specific populations (e.g., hospital admissions for uncontrolled diabetes for American Indian and Alaska Native populations). The committee encourages more comprehensive treatment of the priority populations both within the reports and through other vehicles (e.g., alternate year treatment of priority populations in the reports, spinoff mini-reports with additional detail, customization of data via NHQRDRnet). The national reports should convey key measures that address top health concerns of the priority populations if they are not already part of the AHRQ set of core measures; inclusion would depend on passing the same rigorous evaluation process for measures outlined in Chapter 4.

Given the limitations in the length of a print version of the NHDR, other vehicles can provide additional opportunities for more in-depth treatment. Specialized products for audiences interested in specific priority populations may garner more attention than solely expanding the priority population sections within the NHQRs and NHDRs. These derivative products could be published over time, perhaps in conjunction with partners who have a particular interest in care related to a topic or population.

While "women," "children," and the "elderly" are priority populations, they do not belong solely in the NHDR. At a minimum, the committee believes a summary of findings for these populations should be available in the NHQR. Moving to the NHQR much of this material, which is now in the NHDR, would open up space in the NHDR. The committee expects further inclusion of children's quality measures in the future as a result of the findings from AHRQ's National Advisory Council Subcommittee on Quality Measures for Children in Medicaid and Children's Health Insurance Programs, and the ongoing AHRQ- and CMS-funded IOM study of Pediatric Health and Health Care Quality Measures (AHRQ, 2009c; IOM, 2010). The full number of metrics and the various analyses that might be performed will likely exceed the capacity of the print NHQRs and NHDRs; as noted earlier, more detailed treatment could be accomplished through a special topic report, alternating in-depth sections in the NHQR or NHDR, and/or an ability to customize reports through Web-based capabilities.

Bridging the NHQR and NHDR

An inherent problem in having two separate reports is that data on a subject (e.g., cancer, heart disease, a priority population) are split between the NHQR and NHDR. Moreover, in the NHDR, data on a subject are often hard to find because information is dispersed between different sections (go to Box 6-1). This fragmentation of information will continue to exist, but adding an index at the end of each book would help users find information more readily.

The committee also notes that introductory page(s) for the same topic in the NHQR and NHDR tend to cover the same types of information but are laid out differently; this requires unnecessary effort on the part of AHRQ staff and leads to confusion by readers. For example, the pages on effectiveness of cancer screening in the NHQR (AHRQ, 2009e, pp. 32-33) and NHDR (AHRQ, 2009d, p. 39) contain similar yet not identical information. The committee finds it logical to convey the same information in both locations.

Box 6-1. How Do I Find Disease-Specific Information in the NHDR?

When examining a health topic or specific population in the NHDR, the information is often difficult to find. For example, if one aims to look at colorectal cancer, one is unable to access all of the information about colorectal cancer in one place.^a The report is organized by sections, and all but one (Access to Health Care) contain some information on colorectal cancer. (Go to the following sections in the 2008 NHDR: Quality of Health Care, pp. 39-43, 46, 122, 127; and Priority Populations, pp. 177-181, 185, 188-189, 199, 201, 203, 207, and 211 [AHRQ 2009d]). These pages include information on screening, mortality, and differences by race and socioeconomic status. Additional data on colorectal cancer are available in the NHQR.

Where should consumers, policy makers, or health services researchers go to access sought-after information? Should one search in the Quality of Health Care section or the Priority Population section of the NHDR? Without an accompanying index, one must sift through the entire report to find all disease-specific information. All of the information on a single topic could be linked through an online search tool within the existing NHQRDRnet (which now only supports data tables but not text), other search tools, and linkages between sections in the NHQR and NHDR on the main report site. Even with linkages, the multiple areas in which a single topic appears within the NHQR and NHDR may not be intuitive to readers.

^a Colorectal cancer and breast cancer are reported in alternate years in the NHDR.

Expanding and Sustaining Interest Through Derivative Products

AHRQ provides online access to the national healthcare reports and State Snapshots, to a few report-related fact sheets, and to an online data query system (NHQRDRnet); the Future Directions committee suggests changes to each of these products (go to Table 6-1).

Fact Sheets and Mini-Reports

AHRQ has previously developed three fact sheets to supplement the NHQR and NHDR. The fact sheets have addressed the subjects of children and adolescents (AHRQ, 2005b, 2008a, 2009b), women's health (2005c), and rural health (2005a). These fact sheets are not easily accessible on the national healthcare report-related Web sites; instead they are listed on AHRQ's Measuring Quality Web site.

Concise fact sheets are a way to expand AHRQ's reach of NHQR and NHDR findings and are useful for reaching new audiences. Timing the release of fact sheets to specific events (e.g., heart disease or breast cancer awareness months) could help sustain interest in the national healthcare reports throughout the year. Currently, Internet traffic to the NHQR and NHDR tends to decrease about two months after the report release date. Periodic releases of fact sheets could direct Internet traffic to the reports.

There may be times when a derivative product elaborating on a specific topic (e.g., a mini-report) could provide information beyond what can be contained in a fact sheet. The committee believes that such mini-reports could provide expanded treatment of priority populations. Priority population-specific derivative products would allow fuller exploration, for example, of the particular health care concerns of a priority population (e.g., children, women) or the diversity of health care experiences of different granular ethnicity groups within a race category (e.g., the Asian American population, for instance, is made up of persons of Japanese, Korean, and Cambodian heritage, among other granular ethnicity groups).

State Snapshots

In 2006, using data collected for the NHQR, AHRQ created the Web-based State Snapshots to fulfill the needs for state-level information of members of Congress, state officials, health care providers, and purchasers. As noted by previous IOM guidance, "analyses such as state-by-state comparisons on health care are familiar and meaningful to members of Congress, other policy makers, and consumers" (IOM, 2002, p. 5). The committee finds AHRQ's Web-based State Snapshots to be a valuable addition to the NHQR and NHDR and recognizes recent improvements to the State Snapshots Web site. Nevertheless, the committee urges further development. For example, the State Snapshots do not show any data on access measures, and the committee believes these data are important to have at the state-level.

Health care report cards provide information about the quality of care by geographic regions, health plans, hospitals and other institutions, and even individual practitioners (Epstein, 1995). Report cards use various systems of scoring and passing judgment on quality, whether the end result is to grade national health performance, rank a state's health care quality against all others, compare head-to-head the quality of care delivered in cities across the country, or to develop a list of best value hospitals (Brooke et al., 2008; Chernew and Scanlon, 1998; Davies et al., 2002; Hibbard and Jewett, 1997; Romano et al., 1999). A 2006 qualitative study conducted by AcademyHealth indicated that users of the State Snapshots suggested a rank ordering of states so that states could compare their performance against all others (Martinez-Vidal and Brodt, 2006). Currently, in the State Snapshots, each state is ranked on 18 selected measures. The committee's view is that state-by-state ratings should be more clearly available so that states know what the best attained level of quality performance is, and then they could contact and learn from states with the best rates on specific quality measures. Additionally, it would be useful if state data and rankings were easily sortable for high-profile sets of metrics such as AHRQ core or HEDIS (Healthcare Effectiveness Data and Information Set) measures, given the almost 200 measures that AHRQ tracks for states.

AHRQ displays average regional performance on measures in the State Snapshots, but state audiences have indicated that adjoining states are not always peers. AHRQ has provided a graphical "dial" to show states how they fit on a spectrum of contextual factors (i.e., demographics, health status, etc.), but states have noted that they would like flexibility to be able to identify a different coterie of peer states (for example, states that have the same degree of contextual factors). Then, for example, a state policy maker could assess state performance against states that have a comparable extent of persons below the poverty level.

NHQRDRnet

Users of the NHQR and NHDR suggest that their needs for information tend to be topic specific and episodic; most users of the national healthcare reports are unlikely to read the reports from cover to cover. Additionally, the reports are viewed and downloaded online more than they are used in hard copy.² Thus, improving the ability of users to find needed information online is an important aim. In 2008, AHRQ added an online interactive tool called NHQRDRnet that can be used to query and search the databases behind the NHQR and NHDR by content areas (quality, access, patient safety, priority populations), clinical conditions, care type or settings, and dimensions of access (e.g., insurance coverage, usual source of care, utilization).

The committee applauds AHRQ's intent to facilitate searching for content in the national healthcare reports but finds navigating the NHQRDRnet Web site difficult. The committee also observes that it takes fewer steps to gain similar information from the more straightforward and easier to use Appendix D of the NHQR and NHDR. AHRQ recently commissioned a usability survey that queried current users of the national healthcare reports about their experience with and impressions of the NHQR and NHDR and related Web content (Social & Scientific Systems and UserWorks, 2009). Comments from survey participants regarding ease of using the Web site and clarity of information echoed the committee's findings (e.g., difficulty finding the reports using a basic Web search, organization of report, and Web content not matching user expectations).

The major change the Future Directions committee suggests for the NHQRDRnet is the development of a tool or sorting function that would allow users to customize their own reports. Now, one can search for all data related to a topic—for example, cancer—and all data files since 2002 are displayed for download; the search does not yield a fact sheet or summarization of the current content on the subject of interest. At a minimum, links to relevant text of the current year's NHQR and NHDR would enhance the site's usability and ability to tell a more comprehensive story. Additionally, one- or two-page fact sheets or more in-depth mini-reports on topics (whether individual clinical conditions, framework components, or something more specific, such as quality and disparities issues of specific interest to Hispanic persons) would be useful. AHRQ's partnerships with other stakeholders would be assisted by having prepackaged collections of information in its NHQRDRnet index.

Web-based products, in addition to the NHQRDRnet, can be configured to make it easier to guide readers to other AHRQ or non-AHRQ resources that may help with quality improvement. For example, future online versions of the NHQRs and NHDRs could link to interventions highlighted in the AHRQ Health Care Innovations Exchange or other related resources (e.g., CMS, entities utilizing measures for which data at the national level are still aspirational). This linking capacity should also be available through the Web-based version of the NHQR and NHDR without the reader having to go through the NHQRDRnet.

Online Access to Data Used in the NHQR and NHDR

The committee discussed the extent to which the NHQR and NHDR should provide data for geographic areas below the state level. Various stakeholders have noted that the national healthcare reports contain information that is too "high level" for making decisions at the local or practice level. Consequently, the reports may be of less use to some health care providers, local policymakers, or some researchers than if the performance data were stratified to show performance at more local or organizational levels and provided in a timelier manner. State-based data are a unit of analysis that policy makers as well as the public can easily relate to and use for comparative purposes. Given the interest in substate variation (e.g., the Dartmouth Atlas, the University of Wisconsin/Robert Wood Johnson Foundation county-by-county health rankings), these data would be useful to develop over time (The Dartmouth Institute for Health Policy & Clinical Practice, 2010; RWJF and the University of Wisconsin, 2010). The NHQR and NHDR could also include linkages to other HHS data resources on community health status indicators (HHS, 2010).

The data included in the NHQR and NHDR may be reported nearly a year or more after they have been submitted to AHRQ because of the processes involved with compiling data sources, cleaning the data, analyzing it, reviewing the reports at the departmental level, and submitting the work for production. For entities that are evaluating performance in real time (daily, weekly, monthly), such data may have limited use. Still, there are groups that do not have day-to-day access to performance measurement data and would benefit from the wider availability of nationally collected data at a more local level (Kerr et al., 2004). For example, AHRQ has made available county-level data on the number of Hispanic Medicare beneficiaries with diabetes who did not have an eye exam so that one area's community aging agencies could focus intervention efforts (Moy, 2009).

In deciding whether to recommend that AHRQ provide more locally based data, the committee balanced the usefulness of local data, its timeliness, its reliability, and the additional workload for AHRQ staff. AHRQ staff indicated that it is possible to drill down to at least the larger Metropolitan Statistical Areas for about half of the State Snapshot measures, but that smaller Metropolitan Statistical Areas and counties would be more difficult because, for instance, some datasets are likely to require special permissions to present the data in these ways.³ The committee encourages AHRQ to explore the feasibility and value of drilling down for at least some high-impact measures. When summarized in the reports, the Highlights section, or the proposed guide to using the NHQR and NHDR (discussed below), more localized data can inform readers about variation within states. Such detail could be presented in the State Snapshots to show substate variation, particularly when it is readily available in the datasets AHRQ already uses, and perhaps as a derivative product similar to the Atlas of Mortality (Pickle et al., 1996), depending on the availability of data for coverage of the United States. Links could be made to the HHS Community Health Status indicators site if it eventually included health care-related metrics and not just health status.

Individuals wanting to work with primary data are often not satisfied with the data available through the national healthcare reports' Web site. AHRQ provides Excel files with the data points reflected in its graphs and text, but it does not provide access to the original datasets. Although AHRQ does not have in-house all of the databases it uses in the NHQR and NHDR, most of the data AHRQ uses are from federally sponsored datasets. The committee believes that data access could be expanded so that researchers can download the full dataset to manipulate it as needed. This is consistent with the efforts of data.gov, a Web site currently under development that will house all federal executive branch datasets, to "increase public access to high value, machine readable datasets generated by the Executive Branch of the Federal Government."⁴ AHRQ is among the agencies contributing data, as are other federal agencies whose data AHRQ acquires (e.g., the Centers for Disease Control and Prevention, CMS). Because the Web site is still under construction, the committee is unable to discern which of AHRQ's datasets will be made available. Nonetheless, the committee feels that making available datasets that support findings in the NHQR and NHDR would be a service to providers and health services researchers.

The committee encourages AHRQ and its partners to provide access to the data in a timely fashion, even prior to its publication in the NHQR and NHDR, to allow those with the capacity for analysis to use it for their own needs. Such data access was previously recommended to AHRQ by the IOM in the 2001 report Envisioning the National Health Care Quality Report. Further, AHRQ should provide tools for analysts who want to replicate AHRQ's methods to produce comparative data for their locale or population cohort. Until such tools are available, analytic methods will need to be clearly specified in methodology descriptions.

Proposed Development of a Guide to Using the NHQR and NHDR

Given the diversity of resources that AHRQ now offers and the potential for greater direct data access, the committee suggests that AHRQ develop a guide to using the NHQR and NHDR. As envisioned by the committee, this technical assistance product would review the resources that the print NHQR and NHDR and Web sites have to offer and, more importantly, would provide examples of how different stakeholder groups can apply the knowledge to action (e.g., Hispanic elders diabetes project). The guide to using the NHQR and NHDR would go beyond telling someone how to navigate a Web site. Instead, it would tells users how to access the data resources, provide tools for manipulating data for analyses, explain the methods used by AHRQ in its analyses, and offer suggestions for meaningful analyses. This guide, when it becomes available, should be referenced in the Highlights section of the NHQR and NHDR.

Dissemination Strategies

The committee proposes communicating the findings of the NHQR and NHDR to diverse audiences through a series of new products and modifications to existing documents. The goal of expanded dissemination efforts should be to raise awareness, visibility, and use of the reports. Between 2003 and 2008, AHRQ distributed approximately 24,000 print copies of the NHQR and NHDR.⁵ The annual release of the NHQR and NHDR should be more widely publicized in advance, and momentum from the release of the reports should not be permitted to dwindle. The committee sought input on dissemination and media strategies for the NHQR and NHDR, as well as sample fact sheets, from Ketchum, a public relations and communications firm.⁶

Ketchum suggested ways to repackage the wealth of information contained in the NHQR and NHDR throughout the year so that findings can be made continually relevant. AHRQ could, for example, produce succinct derivative materials that convey targeted messages (e.g., mini-reports and fact sheets), and link distribution and media outreach to appropriate audiences (e.g., advocacy groups for specific clinical conditions or population groups). In addition to relying on traditional media outreach (e.g., participating in roundtables, telebriefings, radio media tours, outreach around editorial calendars), AHRQ could take advantage of wide-reaching and increasingly common Web-based tools (e.g., blogs, advanced search engine options, inbound linking programs, social media such as Facebook and Twitter).

In Chapter 5, the committee emphasizes that a patient's language need is relevant to health care quality and disparities. Communicating information about quality and disparities in languages other than English is one way to reach diverse audiences. The committee acknowledges the potential resource burdens of document translation and does not intend for AHRQ to translate the entire NHQR and NHDR. When derivative products are specifically geared to audiences that may have limited English proficiency, AHRQ may wish to consider translation. HHS' Office of Minority Health Resource Center provides technical assistance in communications and outreach to other HHS agencies (HHS, 2009b). As AHRQ expands dissemination of the reports and their derivative products to diverse stakeholder audiences, the Health Resource Center may provide valuable capacity-building support.

¹ Healthcare Research and Quality Act of 1999, Public Law 106-129, 106th Cong., 1st sess. (November 19, 1999).
² Personal communication, Farah Englert, Agency for Healthcare Research and Quality, November 12, 2009.
³ Personal communication, Ernest Moy, Agency for Healthcare Research and Quality, November 18, 2009. AHRQ staff estimate that data for large Metropolitan Statistical Areas (MSAs) could be provided for more than 50 percent of the State Snapshots measures. All CMS Compare systems could support MSA-level estimates. The Behavioral Risk Factor Surveillance System (BRFSS) could be used for MSA-level analyses for the top 150 MSAs. Other data sources, such as vital statistics data and hospital data from Healthcare Cost and Utilization Project (HCUP) states, would require special permissions in order to analyze and publish data at the MSA-level.
⁴ The data.gov Web site is accessible at http://www.data.gov/ (accessed December 12, 2009).
⁵ Personal communication, Farah Englert, Agency for Healthcare Research and Quality, November 16, 2009.
⁶ Copies of the sample media and communication plan and sample fact sheets developed by Ketchum were provided directly to AHRQ staff and archived in the IOM public access file for the Future Directions project.

Return to Contents
Proceed to Next Section