AQA Invitational Meeting Summary

Report of the Performance Measurement Workgroup

Kevin Weiss, American College of Physicians

Kevin Weiss opened his remarks by noting that although the Performance Measurement Workgroup does not have a perfect process, it does have one that is robust, transparent, and self-effacing. He briefly reviewed past accomplishments, including the following:

• Approving the AQA Parameters for Selecting Ambulatory Care Performance Measures.
• Approving a 26-measure AQA ambulatory care starter set.
• Beginning to enlarge the measure set to include cardiology, cardiac surgery, and patient experience in ambulatory care (ACAHPS) measures (for a total of 86 measures, counting ACAHPS as a single measure).
• Approving the Guide for the Selection of Performance Measures for Medical Subspecialty Care.
• Anticipating that registries will be a part of the discussion and approving Principles in the Use of Registries for Enhancing Quality of Care Through Performance Measurement.
• Approving the AQA Principles of "Efficiency" Measures.

Next, Weiss outlined his workgroup's meeting objectives:

• Review and approve the proposed list of prioritized conditions/procedures for cost-of-care efforts.
• Review the cost-of-care measures discussion paper.
• Review and approve the quality metrics for emergency medicine, gastroesophageal reflux disease (GERD), hematology, and oncology.
• Review other topics under development and get feedback on how to prioritize a long and robust set of ideas.

Cost-of-Care Measurement

Opening the discussion on cost of care, Weiss said that in 2007 the workgroup hoped to do the following:

• Finalize and gain approval for a cost-of-care principles document.
• Work with measure developers to see the priority list of conditions for cost-of-care work turned into measures.
• Further refine the discussion around total cost-of-care measures.

He then briefed participants on how the Prioritized List of Cost-of-Care Conditions had been developed. Weiss said that the Cost-of-Care Workgroup had approved with an "overwhelming consensus" a proposed starter set of 20 conditions and recommended that the list be referred to the AQA Performance Measurement Workgroup for review and approval. The Performance Measurement Workgroup then reviewed the conditions. Using a modified Delphi process, the workgroup engaged in two rounds of voting and discussion to arrive at the selected seven prioritized cost-of-care conditions.

The prioritized conditions address the following:

1. Diabetes.
2. Acute myocardial infarction.
3. Congestive heart failure.
4. Coronary artery disease.
5. Asthma.
6. Depression.
7. Low back pain.

Weiss noted that six of the seven conditions were approved with overwhelming majority support. He said the key criterion for inclusion was that there had to be quality metrics associated with each condition so that cost and quality could be measured. Weiss noted, however, that there was some discussion within the workgroup as to whether the measures were too narrow (three address cardiovascular care) and also as to whether there were quality metrics for depression.

The workgroup addressed the seventh condition, low back pain, separately. Although low back pain did not have quality measures endorsed by the AQA (and thus no quality metrics in place), the workgroup ultimately decided to include it on the priority list. Weiss said that most workgroup members approved the decision, although there were two "no" votes and a few abstentions.

Following Weiss' remarks, there was considerable discussion about whether to include low back pain. A number of people argued that low back pain should be taken off the list because of the lack of quality measures.

Others spoke in favor of keeping low back pain on the prioritized list. Several participants noted that low back pain impacts a wide range of patients. They also noted that there is enormous variation in the care and treatment of low back pain across a number of specialties. Several others stressed that low back pain is an enormous problem for the purchaser community. One person noted that low back pain is generating significant costs and quality concerns. She added that if the AQA did not address the condition, then various health plans and employers will start to study the issue on their own.

Finally, two people pointed out that metrics are available on low back pain. One said that a set of low back pain measures is in operation in Ontario and that this program can be studied to see if it is feasible. Another said low back pain can be measured on the basis of outcomes, such as activities of daily living.

There was also discussion of other items included on—and left off—the prioritized list. One participant noted that there are no robust sets of measures for depression, asthma, or some of the other conditions on the prioritized list and that there is a lot of uncertainty over whether we can develop measures of cost of care and whether they can be appropriately applied to individual physicians.

One participant wondered why cancer is not on the prioritized list. In response, Weiss indicated that, although three types of cancer (prostate, breast, and colon) had been on the initial candidate list of conditions, the quality metrics were not as crisp as they were for those conditions that did make the priority list.

Several participants expressed concern that 24 months was too long a timeframe for developing cost-of-care measures and that a more aggressive timeline was needed. In response, Weiss said that moving faster was really resource-dependent. He added that the Quality Alliance Steering Committee and the NQF's Efficiency Workgroup are also addressing cost of care.

A representative from the NQF noted that one of the reasons the priority list of conditions was expanded to seven items (from an initial plan to target five) is that the NQF is already looking at two of the measures. She said that the NQF process is studying what aspects of performance (both quality and cost) should be assessed given the evidence base and gaps, and will also consider safety and patient engagement on decisionmaking. She said that the NQF hopes to complete the process for at least the first two conditions within 1 year.

One person asked for clarification as to whether the goal was to look at diseases across settings or at the whole patient with a disease or with a disease across the entire continuum of care. Weiss replied that the workgroup has not yet gotten to that level of specificity but that he suspected the NQF workgroup would address these questions. He added that the discussion at this meeting was focused on disease but that the measure developers could later focus on the details.

Finally, someone noted that quality and cost-of-care measures must go hand in hand and that the first question to be asked about cost-of-care measures would be "How do we know what the impact on quality is going to be?" In response, Weiss stressed that the AQA was not being asked to vote at this meeting on cost-of-care measures but only on prioritization of areas for development of cost measures.

Before any votes were taken, there was discussion about whether to vote on the priority conditions en bloc or one at a time. One person suggested a motion to remove low back pain from the list. Another suggested a separate vote on low back pain. Another argued for an en bloc vote, saying that none of the conditions should be separated out.

There was also considerable discussion about how to manage the voting process and who would be eligible to vote. Votes for this meeting were by recorded show of hands (where necessary) and based on the principle of one vote per organization.

Before the debate over how to vote on low back pain, Carolyn Clancy raised the issue of the AQA's voting procedures. She said that the AQA Steering Committee would bring a recommendation on voting procedures for future meetings before the AQA at its next meeting.

Motion: To endorse the Prioritized List of Cost-of-Care Conditions, excluding low back pain. Result: The motion was adopted with near unanimity.

Motion: To include low back pain within the Prioritized List of Cost-of-Care Conditions. Result: The motion was adopted, 53-24, with 4 abstentions.

Before moving to the next agenda item, one person stressed that the goal of the AQA has been to move performance measures forward as quickly as possible, pointing out that AQA must be able to move an agenda and give every stakeholder confidence that there is a clear path forward with a defined timeline. He added that AQA needs to figure out how to get the funding necessary to move an agenda that is precise, is clearly defined, and contains very clear timelines.

Regarding timelines, Weiss noted that AQA is moving quickly—there hadn't even been any discussion about cost of care a year ago—but to move forward even faster, more resources would be needed.

Next, Weiss introduced the draft Cost-of-Care Measurement Set document. The purpose of the document is to define the objectives of cost-of-care measures and the key components to help ensure reliable measurement. The document notes that the Performance Measurement Group recognizes the need for:

1. Measures that may be related to specific conditions or procedures.
2. A composite measure of the relative cost-of-care performance of a physician's total practice.

The document also details anticipated uses for cost-of-care measures, including the following:

• Performance education and engagement.
• Clinical performance improvement.
• Performance incentivization.
• Transparency and consumer choice.
• Iterative improvement of measurement tools.

In addition, the document addresses:

• Measurement systems and the need for transparency.
• The need to present cost-of-care measures, wherever possible, as average costs compared with expected costs.
• Principles and procedures for prioritizing and developing condition- or procedure-specific starter sets.

The paper will be brought forward for a vote at the next AQA meeting.

Following Weiss' remarks, two participants said they were glad that language addressing feedback and integrity ("there should be a formal feedback and correction mechanism so that errors uncovered by physicians, plans, and other analysts can contribute to improving the evaluation system") had been included.

Another person strongly endorsed the document, saying that it would provide great guidance to employers, who are looking for tools like this.

Proposed New Quality Measures for Adoption

Kevin Weiss discussed new quality measures for:

• Emergency medicine.
• Gastroesophageal reflux disease (GERD).
• Hematology.
• Oncology.

Weiss said that, when the workgroup was considering the measure sets, two crosscutting issues came up:

• The issue of entry or competency measures versus high-performance measures.
• The issue of exclusions.

Weiss elaborated on the first issue, saying that it was important to talk about the spectrum of measurement and that no threshold has been put on any measures yet, as there was not yet a framework for doing that. Weiss asked participants to set aside the issue to vote on the measures today, saying that he would seek to address the matter either through the AQA Performance Measurement Workgroup or in partnership with another pertinent forum.

Emergency Medicine

Proposed Measures Accountability: Electrocardiogram performed for non-traumatic chest pain. Aspirin at arrival for acute myocardial infarction (AMI). Electrocardiogram performed for syncope. Vital signs for community-acquired bacterial pneumonia. Assessment of oxygen saturation for community-acquired bacterial pneumonia. Assessment of mental status for community-acquired bacterial pneumonia. Empiric antibiotic for community-acquired bacterial pneumonia. Quality Improvement Only: Fibrinolytic therapy ordered within 20 minutes of electrocardiogram performed for AMI. Care coordination for percutaneous coronary intervention for AMI.

In introducing the proposed emergency medicine measures, a participant representing the American College of Emergency Physicians noted that the measures are currently under consideration in the NQF.

Motion: To approve proposed emergency medicine measures No. 1-7. Result: The motion was adopted with near unanimity.

Gastroesophageal Reflux Disease (GERD)

Proposed Measures Accountability: Assessment for alarm symptoms. Upper endoscopy for patients with alarm symptoms. Biopsy for Barrett's esophagus. Barium swallow—Inappropriate use. Quality Improvement Only: Medication therapy—Assessment of GERD symptoms.

In introducing the proposed GERD measures, a participant representing the American Gastroenterological Association noted that the measures have been through the Physician Consortium for Performance Measurement process. He said they were subsequently submitted to the NQF and have been reviewed by an NQF technical advisory group.

Weiss indicated that there had been considerable discussion within the Performance Measurement Workgroup about whether the GERD measures should be assigned a specialty. He said there was also a lot of discussion around measure No. 1 (assessment for alarm symptoms). Ultimately, all but a few people voted to report out the entire measure set.

Following Weiss' remarks, one participant raised the issue of primary care. He said that the issue of how expansive the measure set will be needs to be addressed. What will primary care physicians be accountable for and need to report on? Another participant expressed concern that the notation regarding "adequate data sources only if new codes are developed" would preclude the use of administrative data.

Motion: To approve the four proposed GERD accountability measures. Result: The motion was adopted with near unanimity.

Hematology

Proposed Measures Accountability: Myelodysplastic Syndrome and acute leukemias—Baseline cytogenetic testing performed on bone marrow. Myelodysplastic Syndrome—Documentation of iron stores in patients receiving erythropoietin therapy. Multiple myeloma—Treatment with bisphosphonates. Chronic lymphocytic leukemia—Baseline flow cytometry.

In introducing the proposed hematology measures, a participant representing the American Society of Hematology outlined the development process. He said that the measures have gone through the Consortium process and are ready to be submitted to the NQF.

Weiss noted that the workgroup had discussed whether or not these measures were too entry-level and whether they fit in well with standards of care. He said the workgroup ultimately endorsed the hematology measures with few "no" votes or abstentions.

Motion: To approve the four proposed hematology measures. Result: The motion was adopted with near unanimity.

Oncology

Proposed Measures Cancer staging documented in the medical record. Hormonal therapy for stage IC-III estrogen receptor/progesterone receptor positive breast cancer. Chemotherapy for stage III colon cancer patients. Plan for chemotherapy documented before chemotherapy administered. Radiation therapy for invasive breast cancer patients who have undergone breast conserving surgery.

In introducing the proposed oncology measures, a participant representing the American Society of Clinical Oncology (ASCO) said her organization has a long history of developing measures. She added that ASCO had a library of more than 50 measures tested in 100 oncology practices around the United States.

Weiss acknowledged that ASCO's path to developing measures is different from the multistakeholder process that the AQA encourages. He said that, in voting to bring the oncology measures to the full AQA, the workgroup understood that the Consortium would take up the measures (due to happen in late February) and that the measures would then be submitted to the NQF. In addition, the amount of predevelopment work and the substantial and robust vetting process that had taken place within ASCO convinced the vast majority of workgroup members to vote "yes." He said measures No. 2 through No. 5 were voted out of the workgroup with very few "no" votes.

One person raised concerns about approving the oncology measures, saying that having a consensus process with multiple stakeholders at the table is crucial. Another person noted that measures No. 2 and No. 3 address what should be prescribed. He suggested that, where appropriate, measures should look not only at what is prescribed but also at whether a prescription has been filled. Although this second element isn't completely under a physician's control, it is the responsibility of physicians to engage their patients to comply.

Another participant noted that measures No. 2, No. 3, and No. 5 have made their way through the NQF process. Speaking in support of approval, he indicated that the oncology community would like to have something to report on. A participant representing the NQF clarified that the three measures had not yet been endorsed by the NQF. She added that the NQF has put them forward as potential system-level measures (not clinician-level measures).

Motion: To approve proposed oncology measures No. 2-5. Result: The motion was approved 42-22, with 6 abstentions.

Turning to measure No. 1, Weiss indicated that it had been the most controversial within the workgroup for two reasons. First, it was not developed using the same process as the other measures; rather, it was added later. Second, there were concerns that the measure was too basic.

One participant wondered whether measure No. 1 was a performance measure and had been created merely so the other ones could be implemented. In response, the ASCO representative said that the data suggest that not all physicians are currently documenting the staging information. Another participant said that, although the measure is very basic, it identifies a quality gap. A third participant spoke in opposition to the measure because it had not been vetted through a multistakeholder process.

Motion: To approve proposed oncology measure No. 1. Result: The motion failed.

Work in Progress

Next, Weiss turned to the workgroup's extensive to-do list for 2007. He reminded people to review the Cost-of-Care Measurement Set document and provide any feedback as the workgroup intended to bring the document forward for a vote at the next AQA meeting. He also asked participants to review the draft AQA Principles for Appropriateness Measures document (designed to provide guidance to measure developers on how best to construct implementable appropriateness measures that will meet multistakeholder needs). In addition, Weiss noted that the workgroup is developing a test measure designation as well as principles for AQA test measures (and what that designation might mean), adding that they would like to field-test measures before the measures must be reported on publicly.

Weiss also highlighted other items on the workgroup's 2007 agenda:

• Continuing to review measures submitted by medical and surgical specialty societies.
• Pursuing measurement on the prioritized list of cost-of-care conditions.
• Pursuing measurement on composite measures for total cost of care.
• Integrating the workgroup's efficiency measures work with that of the Quality Alliance Steering Committee and the NQF.
• Discussing when measures can be studied on an individual vs. physician group level.
• Developing principles around measure implementation/data sourcing (including consideration of principles around exclusions).
• Beginning to discuss prioritization of focus on specialty measure development.
• Considering developing principles of use of measures along the competency/high-performance spectrum (and setting the criteria to allow the AQA to determine and even possibly tag where on the spectrum a measure falls).
• Considering principles in the use of measurement as related to special populations.
• Considering quality and cost harmonization.
• Developing principles for composite quality measures.
• Encouraging the development of ACAHPS measures for surgery and other population needs (including pediatrics).

Opening the discussion on prioritizing the agenda, one participant suggested that the place to start was with the things we know are going to happen and cross-correlate those with what physicians can tolerate. He also suggested laying out where the gaps are. In response, Weiss said that there was a pretty clear picture of how the development process was proceeding within both the Consortium and the NQF. The missing piece, he said, is "measures that are okay but not really what we need." He added that there was also a need to prioritize the focus on specialty measure development.

One participant expressed concern about setting priorities. What aren't we going to do? Everyone in a multistakeholder process has different priorities.

Weiss said that one approach would be to look at and get a better understanding of where some of this work is already being undertaken within the broader health care community.

Another participant suggested that the workgroup ought to first map out how the various initiatives might be staged. She pointed out that the AQA offers a unique opportunity to have a conversation among multiple stakeholders about how to think about quality going forward. This follows a model that has worked well up until now (measures developed by specialty societies, brought to the Consortium, and then brought to the NQF).

The same participant also touched on the issue of the spectrum of competency and whether or not this should be left to boards and licensing bodies to address.

One participant raised a question regarding individual vs. physician group analysis. Will CMS be committed to collecting these data by provider identifier number (and not just by tax ID number)? In response, a CMS official indicated that the agency was committed to addressing the issue and trying to get some answers.

Another participant suggested that the AQA needs a mechanism for identifying a measure's developer, knowing whether the measure has been endorsed by the NQF, and knowing whether the measure has been approved by the AQA. He pointed out that it was very important for those in the field to figure out the status of a measure and that this was not possible today. The same participant also raised two other issues:

1. The importance of understanding the disease burden of physicians (which is different from the specialty).
2. The need to map out measures that are appropriate for physicians' practices.

One participant spoke of the need to have a strong emphasis on more meaningful, outcome-oriented, high-performance measures. He added that he strongly supported the view that the workgroup should focus on populuations at risk and known gaps in care.

Another participant pointed out the need to start to coordinate multiple initiatives/groups and to define the strategic needs of the patient in health care. Patients want to be able to find a quality physician. In addition, he said that more strategic networking is needed outside the AQA process in order to create a learning network to start to close gaps in quality care.

The same participant warned against measure overload and stressed the danger of creating a lot of measures (because of the incentives for reporting) that don't align in strategic ways. Referencing the agenda item on prioritizing focus on specialty measure development, he said it was important to develop a measure that addresses how to deliver care to patients regardless of setting—rather than measuring how physicians do this—and that the effort must include key initiatives in primary care so that the strategic needs of patients are met without putting an undue burden on physicians.

Several other participants also stressed the need for a patient-centric approach. One participant noted the need to drive high-performance measures that are meaningful for the consumer and the need to prioritize efforts around health disparities.

The next step, according to one participant, is to embed the AQA's work into everyday work. Summing up the discussion, Weiss said the message that each of the agenda items is important was clear. He said it was also clear that the workgroup needed to place a little more emphasis on a few agenda items, and he promised to bring those back for discussion at the next AQA meeting.

---

Previous Section        Contents         Next Section

---