Future Validation and Improvement of the AHRQ QI (Text Version)

Slide Presentation from the AHRQ 2008 Annual Conference

---

On September 10, 2008, Kathryn McDonald made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (45 KB).

---

Slide 1

Future Validation and Improvement of the Agency for Healthcare Research and Quality (AHRQ) Quality Indicator (QI)

AHRQ Annual Conference
September 10, 2008
Bethesda, MD
Presented by Kathryn McDonald

Slide 2

Validation: Where we have been

• AHRQ QI team activities:
  • Face validity of indicators through panel review.
  • Coding validity through chart review of selected Patient Safety Indicators (PSIs).
• Outside research:
  • Present on Admission (POA).
  • Coding validity.
  • Resource use.

Slide 3

Validation: Where we are headed next

• Continued chart reviews:
  • More PSIs, estimating false positives.
  • Examination of under-reporting of PSIs.
  • Creation of chart review tools.
• Prevention Quality Indicators validation:
  • Extending the face validity of the indicators through panel review.
  • New tools for using the PQIs.
• Additional work on Patient Safety Indicators:
  • Assessment of new coding and POA.
• Additional work on Neonatal Indicators/Pediatric Indicators (PDIs):
  • Assessment of coding validity for bloodstream infection (BSI).
  • Risk adjustment development and validation.

Slide 4

Our Progress Depends on Collaboration

• Continuous feedback loop from validation work and user feedback creates better indicators.
• Spans all types of research projects:
  • Peer reviewed to individual hospital investigations.
• Informs potential feedback on coding, guidance on indicator use, and in some cases indicator reassignment.
• Motivates additional validation studies.

Slide 5

Research and Use Cycle

• Graphic depicts continuous cycle:
  • QI User Feedback and Validity Research.
  • Assess the Generalizability of the Experience or Findings.
  • Design and Test Indicator Improvements.

Slide 6

Examples of Cycle Benefit: Indicator Reassignment

Complications of Anesthesia:

• Several user reports noted that minor reactions such as pruritis were coded.
• Investigation of coding guidelines found that E-codes used in this indicator allowed for coding of these minor reactions.
• No way to "fix" indicator found.
• Indicator will be reassigned as an "experimental indicator."

Slide 7

Examples of Cycle Benefit: Modifying Guidance

Present on Admission Research:

• Several recent studies have highlighted POA rates in the Patient Safety Indicators.
• Revealed indicators for which POA is an important data element.
• Guidance to use these indicators only with POA, software modified to require POA in some cases, and National Quality Forum (NQF) endorsement conditional on POA.

Slide 8

Examples of Cycle Benefit: Modifying Coding Structure

Transfusion Reaction:

• Intricacies in coding aren't always obvious in ICD-9-CM coding.
• National Association of Children's Hospitals and Related Institutions (NACHRI) supported research of the PDIs identified cases of transfusion reaction from minor antigens instead of ABO.
• Minor antigens indexed to ABO code.
• Proposal to separate transfusion reactions due to ABO from those due to minor antigens to improve the specificity of this indicator.

Slide 9

Examples of Cycle Benefit: Improving the Indicator

Respiratory Failure:

• Initial definition relied on diagnosis code for identifying numerator cases.
• VA based study found low sensitivity for dx code.
• Further investigation by Department of Veterans Affairs (VA) team identified procedure codes for delayed extubation and post-operative re-intubation improved sensitivity without significantly decreasing specificity.
• AHRQ team investigated generalizability of these findings using the Healthcare Cost & Utilization Project (HCUP) data and...
• Added the new procedure codes to the definition.
• But the story doesn't end there...

Slide 10

Examples of Cycle Benefit: Improving the Indicator (continued)

Respiratory Failure (continued):

• Procedure codes were imported into new pediatric indicator.
• NACHRI directed study identified children with expected extended intubations not related to respiratory failure.
• Consulting with experts and through data analysis AHRQ team identified specific operations for which intubation is extended (e.g. tracheal procedures).
• Some cases generalizable to the adult indicator.
• Continued work with NACHRI team to ensure that the solution truly improves specificity of indicator.

Slide 11

The Path for Future Research

• Many user experiences, small and large validation studies happening outside of AHRQ.
• These studies allow us to improve indicators and ultimately hospital quality.
• Tell us about your research on:
  • Validation (sensitivity and specificity).
  • Quality Improvement Programs.
  • Special Populations.

Current as of February 2009

---

Internet Citation:

Future Validation and Improvement of the AHRQ QI. Slide Presentation from the AHRQ 2008 Annual Conference (Text Version). February 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/annualmtg08/091008slides/McDonald.htm

---