Toward an Evaluation of the
Quality Improvement Organization Program:
Beyond the 8^th Scope of Work

Contents

Acknowledgements

Executive Summary (in PDF format, 11 pages)

Chapters

1. Introduction
   1. Growing Interest in Evaluation of the QIO Program
   2. Study Objectives
   3. Organization of Report
2. The QIO Program
   1. History and Background
   2. The QIO Program Today
3. Review of the Literature
   1. Methodology
   2. Evaluations of the Effectiveness of the QIO Program
   3. Qualitative Reviews of the Evolution and Impact of the QIO
   4. The Impact of Individual QIO Quality Improvement Initiatives
   5. Summary
4. Development of QIO Inventory
   1. Sources of Data Information to Conduct the Environmental Scan
   2. Database of QIO Activities
   3. Key Findings
   4. Challenges and Limitations
5. Site Visit Results
   1. Site Visit Methods
   2. Results: Organization and Governance of QIO
   3. Results: Technical Assistance Offered to Providers
   4. Results: Case Review, Beneficiary Protection and Program Integrity
   5. Results: QIO Support Center Functions
   6. Results: CMS Program Management and Evaluation Issues
6. Evaluation Designs & Considerations
   1. TEP Contributions to the Evaluation Design Process
   2. Assumptions, Scope and Organization of Evaluation Designs
   3. Designs for Evaluating the Core Program
   4. Designs for Evaluating the Special Studies Program
   5. Designs for Evaluating Technical Assistance Approaches
   6. Designs for Supporting Poor-Performing and Less Motivated Providers
   7. Designs for Evaluating CMS Performance Targets
   8. Other Areas for Study
7. Recommended First Steps
   1. Inventory CMS Data Systems
   2. Address Limitations in Access to Provider Identifying Data
   3. Maintain Transparency in Designing and Conducting Evaluation

References

Appendices:

1. 8^th SOW QIOs
2. Screenshot of QIO Inventory
3. Selected Studies for the 8^th SOW
4. Technical Expert Panel Biographies
5. Sources of Data for Use in Evaluation

Acknowledgements

Although the authors take sole responsibility for the contents of this report, we would like to thank Laurie Feinberg, our ASPE Project Officer, and Susan Polniaszek who provided thoughtful input and guidance throughout the course of the project. We would also like to thank our subcontractor, Robert Friedland, at Georgetown University, for his many contributions in designing the interview guides, conducting interviews, and providing insight on evaluation designs. Likewise, we very much appreciate the helpful advice and guidance given to us by our study consultant, Shoshanna Sofaer. We gratefully acknowledge the contributions of the members of our Technical Expert Panel who as a group and individually reviewed the evaluation design options and provided us with substantive advice.

Our appreciation goes out to the staff and key informants at each of the nine QIOs that we visited who graciously took of their time to answer our many questions about their work and provided us with a more substantive understanding of the QIO program. In addition, we are very grateful to the staff at CMS and the Institute of Medicine who met with us and provided us with materials to better understand QIO program operations.

Finally, we are very much indebted to our current NORC colleagues Claudia Schur, Caitlin Oppenheimer, Alene Kennedy, Jessica Bushar, Jyoti Gupta and our former NORC colleagues Holly Stockdale, Alana Ketchel, Jennifer Benz, and Jenissa Haidari, without whose hard work and tireless assistance in various stages of this project, including developing the study protocol, designing and populating the QIO Inventory, conducting the site visits, and reviewing reports and other products, this project could not have been completed.

How to Obtain a Printed Copy

To obtain a printed copy of this report, send the title and your mailing information to:

Where to?

Top of Page | Contents

Home Pages:
Health Policy (HP)
Assistant Secretary for Planning and Evaluation (ASPE)
U.S. Department of Health and Human Services (HHS)

Last updated: 09/25/07