AQA Invitational Meeting Summary

Report of the Performance Measurement Workgroup

Kevin Weiss, American College of Physicians

Kevin Weiss thanked the multiple performance measurement workgroups for their efforts, and outlined the AQA's accomplishments to date. These include endorsement of the following:

• Ambulatory care starter set.
• Patient experience of care measures.
• Principles of efficiency measures.

Weiss added that the workgroup's current focus is on the cost of care (efficiency) measures, the acute/chronic subspecialty measures, the surgery/procedures subspecialty measures, and the composite measures.

Weiss outlined his workgroup meeting's objectives:

• Discuss reference to ambulatory care in the parameters document.
• Review and endorse the acute/chronic workgroup's draft guidance document.
• Review and endorse the AQA starter set for cardiology.
• Review the mission and principles of the surgery/procedures workgroup.
• Review and endorse the AQA starter set for cardiac surgery.
• Review and endorse a registry principles document.
• Discuss the cost of care measures draft implementation rules and rationale for prioritizing conditions.

Discussion of Ambulatory Care in the Parameters Document

To begin, Weiss offered a motion to remove ambulatory care from the title of the document AQA Parameters for Selecting Ambulatory Care Performance Measures to reflect the alliance's ever-broadening participant base and scope of activities. He also suggested adding a modifier that includes reference to physician-level measures.

One participant expressed concern that deleting ambulatory care would make the effort less clear. A second participant expressed a broader concern that the process was moving too rapidly and that a lot of stakeholders were not adequately involved. Maybe other organizations have measures that are valid, he said, and these documents may not have been developed with their full input.

In response, Weiss observed that no decision is ever made without the full AQA body seeing every document at least twice. And before it gets to the full AQA, every document is circulated for months.

Motion: To remove ambulatory care from the title of the AQA Parameters document, and to add a modifier that includes reference to physician-level measures. Result: The motion was adopted unanimously.

Weiss noted that, when adding subspecialty measures, the original concept of a starter set would be washed away, unless they are defined as "measures, at least for the short term, that will be focused on that subspecialty."

Acute/Chronic Care Workgroup Report

Bruce Bagley of the American Academy of Family Physicians began his remarks about the guide for selecting performance measures for medical subspecialty care by noting that the document should probably be called the AQA (not Ambulatory Care Quality Alliance) Guide and that in the Ambulatory Setting likely should be deleted from the title. He stressed that the document was intended simply as a guide that laid out some parameters the workgroup thought would be helpful as specialty societies started to develop and promote measures through the AQA process.

These guidelines are recommendations for how this work should be done and what some of the concerns are. He noted, for example, that the guidelines address the absolute necessity to discuss both appropriateness of care measures and variations of care (and, if there is no evidence, then ask the specialty societies to start to develop it). Bagley also referred participants to a proposed starter set of measures for cardiology (Cardiovascular Ambulatory Care Performance Measures, developed by the American College of Cardiology [ACC]).

One participant asked about transitions between sites of care. We need to recognize that there is coordination of care at all sites where patients are, he said. Bagley responded that while the document does not at the moment address that, the workgroup is willing to do so.

There was a question about the appropriateness of care guidelines ("measures that assure appropriateness of care should be utilized, whenever possible"). Bagley replied that it was impossible to move forward focusing only on quality. We must also focus on appropriateness, he said, since our system has some problems with overuse. Weiss added that appropriateness needs to come forward in terms of all areas, and he urged groups to note if they are doing work on appropriateness.

Another participant asked that the document make clear that appropriateness is based on specialty recommendations. It's important that there be a default, she said, to address what happens when measures don't exist.

From the surgical aspect, appropriateness of care is very different than it is regarding use of equipment, said another participant. He added that he was cautious because he would have approached the issue from a different angle than the ACC has taken. Bagley responded that the guidance document does not define appropriateness, but only addresses the need to discuss it.

One participant suggested that measures that ensure appropriateness of care should be included. That doesn't suggest that they're more important, he said, just that they should be included. He also suggested using only the term specialty (and not switching back and forth with subspecialty). Bagley said he thought the workgroup could make those changes.

Another participant noted that most of the guidance was around care after the diagnosis was made and that it would be useful to have measures around diagnosis. We have to work on that, agreed Bagley.

Another participant asked if it would wise to include efficiency here. Bagley replied that efficiency was included under "relevant outcomes" in guideline No. 5. Efficiency is not an outcome, said someone else, so we should find a way to include it in No. 5. This led to a discussion about the document being embedded under the AQA performance measurement rubric. The following questions were raised:

• Is embedding the document under the AQA performance measurement rubric sufficient?
• Does efficiency need to specifically be embedded under No. 5?
• Is a separate guideline on efficiency needed?
• Is a separate guideline needed that asks specialty societies to pay attention to the AQA's other work?

There was also discussion around the language on public reporting:

Measures should consider the importance of good comprehensive care for a condition even if only one or two of the measures will ultimately be used for public reporting. The complete set will be useful for in-office quality improvement efforts.

Why is this here, asked one participant, who expressed concern that specialty groups might have only one or two measures that were reported publicly. In response, Bagley noted that some measures are not appropriate for public reporting and accountability. Another person reminded participants that there are alternatives to public physician-level reporting, including database registries that can provide feedback to members.

Wrapping up the discussion, Bagley suggested that no vote of endorsement be taken on the AQA Guide for the Selection of Performance Measures for Medical Subspecialty Care as there were still a number of concerns with the document. He said that the workgroup would go back to work and, he hoped, return with a document that the AQA could endorse at its next meeting.

Bruce Bagley next turned participants' attention to the Cardiovascular Ambulatory Care Performance Measures (which have been endorsed by the National Quality Forum). He noted that the cardiac surgery measures are fairly consistent with other measures that have been put forward.

Opening the discussion, one participant said he thought that they are fine, as long as the AQA is clear that they apply only to cardiology. He used the analogy of building a temporary fence around different measure sets. He expressed concern that, if these measures aren't segregated (and, likewise, the 26-measure starter set for primary care), then physicians may quickly find themselves having to report on 100 measures rather than taking care of their patients.

Another participant said that while he agrees that there is a potential data burden, we must recognize the need to move this rapidly to the front burner and quickly develop and approve patient-focused measures. He added that whether you are seeing a cardiologist or a general practitioner, as a patient you must be evaluated based on the same measures.

In response, Bagley said that the real issue was that the AQA was using "an imperfect model to get this process moving forward." He asked the specialty societies to bring their performance measures to the AQA, noting that the ACC had brought a bigger set that the workgroup had pared down to ambulatory measures.

It's hard to endorse a principle of "the temporary fence" when we have no language about what that means, said another participant. Bagley agreed that the issue required further discussion.

Another participant expressed concern that the measure set was short on appropriateness and outcomes and didn't really recognize that these measures were likely to be burdensome on physician offices. So the question is, he asked, where do we go next? Bagley stressed that the measures were a starting point only, and that the workgroup had expressed similar concerns.

A participant representing the ACC noted that what was before the AQA were performance measures, not appropriateness criteria. She added that the level of analysis has not been specifically addressed.

Perhaps one of the principles in the previous item should be that the developer identifies specifications for data elements, suggested another participant, so that two people are collecting the exact same data.

Motion: To approve the Cardiovascular Ambulatory Care Performance Measures. Result: The motion was adopted.

Surgery/Procedures Workgroup Report

Frank Opelka turned participants' attention to the AQA Surgery/Procedure Workgroup Mission and Principles document. He said the workgroup intended to capture both multidisciplinary and individual approaches, which are highlighted in principle No. 2:

The measures must be within the control of the physician(s) who perform the procedure. This includes aspects of care within a multidisciplinary team or for an individual physician.

 He added that the document was a work in progress.

One participant asked how the Surgical Care Alliance would fit in with the AQA's multi-stakeholder Surgery/Procedures Workgroup. Is the Surgical Care Alliance's work going to feed into the AQA's work? he asked. In response, Opelka noted that the Surgery Care Alliance was set up to try to better serve and support the surgical societies within the broader AQA process.

A second participant pointed to the importance of principle No. 2, saying that it should be captured in all documents on principles, not just that of the surgery workgroup.

Referring to principle No. 6 (Measures may be procedure-specific or cross-cutting across multiple specialties), a participant asked the surgery workgroup to work on a template that sets forth some examples of cross-cutting measures that might be included in these measure sets. Weiss responded that the entire performance measurement workgroup needed to discuss cross-cutting measures and that he would be happy to bring the issue forward.

Referring to the language on timing in principle No. 3:

Measures developed may occur within the continuum of care, from the moment a decision is made to perform the procedure through the reasonable period of time for associated complications.

One participant suggested that it be changed "from making a decision to perform the procedure" rather than "from the moment a decision is made." The modification was accepted without further discussion.

Motion: To approve the AQA Surgery/Procedure Workgroup Mission and Principles document. Result: The motion was adopted.

Next, Opelka turned participants' attention to the Society of Thoracic Surgeons Cardiac Surgery Measures Starter Set (which has been endorsed by the National Quality Forum). He said that the workgroup was bringing to the full AQA these hospital-based measures that are part of the database system with which physicians are involved.

One participant suggested using the term group practice since the word participant really refers to a group of physicians. This led another participant to ask if the measures were meant to be physician specific. In response, Opelka said that the level of analysis and evaluation of level of care were happening at the group/hospital level.

One participant asked if he was saying explicitly that they do not apply to individual cardiac thoracic surgeons, or if, at this time, most cardiac thoracic surgeons are operating in groups. If so, then would it be appropriate to apply to an individual?  Yes, replied Opelka, it would apply to an individual in private practice.

Another participant expressed concern about applying measures to an entire group of cardiac thoracic surgeons. As a physician who used to refer a lot of patients to such groups, he said he saw a perception and a reality that there are differences in quality within the group. He said he wanted to be able to relay that information to his patients. As a result, he said, while we may have to develop measures that apply to a group setting, we should insist that we also develop measures to discriminate between one physician and another.

The real question isn't whether one physician is better than another, but why, said another participant. Most physicians work with a team, and someone else might not have the advantage of developing another team. It could be a systemic problem and have nothing to do with the individual. He added that it was often hard to make one item or decision attributable to a single physician.

There were a couple of comments about the urgency of moving forward quickly to provide as much information as possible to the purchaser and consumer communities. One participant noted that doing so requires uniformity on performance measurement. If we don't have principles on interrelationships (addressing the segregation issue brought up earlier with regard to the cardiovascular ambulatory care performance measures, as well as the question of group versus individual measures), she said, then we risk having others pass us by. She asked the workgroup if it could work on principles for sorting through these issues to achieve uniformity and bring back guidance at the next meeting.

Another participant noted that, while it was important to be sensitive to fairness and issues of causality regarding measurement of individual physicians, consumers would like to know if there are differences between individual providers.

Finally, one participant noted that the starter set offers a set of process measures together with outcomes. That's very salutary, he said.

Motion: To approve the Society of Thoracic Surgeons Cardiac Surgery Measures Starter Set. Result: The motion was adopted.

AQA Registry Principles

Kevin Weiss turned participants' attention to the AQA Principles for the Use of Registries for Enhancing Quality of Care through Performance Measurement. He noted that the document has undergone changes since it was first reviewed at the January AQA meeting. Weiss took item No. 4 (on condition-specific measures) off the table for the current discussion but asked for comment on the overview and other items.

The discussion opened on the issue of administrative versus clinical data. When one participant said he thought the AQA wasn't going to recommend use of administrative data, Weiss said that the discussion had been on not using clinical data alone. We need to recognize that some physicians might see administrative data as valuable, he said. There was also discussion about whether to change the language, perhaps to read "clinical and/or administrative data."

Concern was expressed about the use of the term registries, with one participant saying she thought it was confusing to discuss registries as a set. She pointed to new, proposed overview language that sought to clarify the term. Carolyn Clancy expressed concern that the revised language was limiting as to what kind of data could be collected. Another participant said he wanted to be sure registries are used only for designated purposes and said it wasn't clear that the alternative language defines that clearly. Yet another participant asked about the potential to use registry data for other areas (such as research and public health surveillance monitoring).

In response to the discussion, Weiss said that he would ask the workgroup to address some of these issues and revisit the matter at the next AQA meeting.

Finally, a participant raised a question about whether item No. 12 (All registries should allow participation by any interested physician within the group that is sponsoring it) was in conflict with earlier items (No. 3 and 4) that are less limiting. He suggested that the concepts needed to be better defined.

Cost of Care Measurement

Turning to cost of care measurement, Kevin Weiss said that the workgroup was starting to explore what implementation rules would be required to look at consistency across a multipayer aggregated data set. He noted that right now, these rules are being used in different ways by different parts of the marketplace. As a result, he said, we want to set up a series of implementation rules for physicians to use for the common reporting of AQA measures. The second part of this discussion, added Weiss, is, once the rules are in place, what are we going to measure? Will it be condition specific, or total cost of care?

He added that the workgroup wants to include a dimension of actionability on cost of care measures. Finally, he touched on the need to address areas of implementation rules (including measure selection, input data, accountable unit, and statistical methods).

Opening the discussion, several participants expressed concern that the Cost of Care Measures Exploratory Task Force's activities did not reflect the work being done by other workgroups. One participant expressed specific concern that the Rationale for Prioritizing Conditions document came out of a meeting that was dominated by industry and had almost no physician participation, while another said that holding conference calls during business hours made it difficult for physicians to participate. In response, Weiss said the issue was very complex and that it was important to create a "physicians' interest group" and get more physicians up to speed so that there is the requisite expertise to participate effectively in this process.

Another participant suggested that two types of physicians should participate in the cost of measurement discussion:

1. Those with experience with episode groupers who know some of the problems.
2. Those new to the discussion who want to understand the process and want it to be more transparent than it has been.

Yet another participant said that nonphysicians interested in cost of care issues should also be brought up to speed.

Cost is driving a lot of our activities, said one participant. We talk about transparency, and this assumes consumers are operating in the free market. He said that physicians are often rated on costs, which leaves them with good ratings only if they provide just the care that payers find acceptable. He said that items not under physician control should not be used for rating purposes.

Another participant asked whether the task force could elaborate on the relationship between cost of care measures and quality measures. Yes, said Weiss, who added that the intent of the task force is to align its work, whenever possible, with quality measures.

One participant said the aim should be cost of care measures that have a quality component (rather than using cost of care measures to get at quality). In addition, he stressed that all stakeholders (not just physicians) should be participating in this process.

Finally, Weiss stressed that the task force was working on a rationale for prioritizing conditions. We want to look at the high-frequency conditions, those with the highest variability, and those with the most cost burden. In that intersection, he said, we can identify a good starter set.

Goals for 2006

Wrapping up his remarks, Weiss outlined the Workgroup on Performance Measurement's goals for 2006:

• Continue to endorse starter sets of surgical and medical specialty care measures.
• Endorse standardized implementation rules for cost of care measures.
• Develop cost of care education tools for physicians.
• Initiate work on composite measures.
• Undertake other work as the marketplace demands.

Finally, Weiss said he wanted to establish a formal working group discussion on unintended consequences: what might be the nature of unintended consequences, and how to monitor for them and adjust the system as needed. We need to do what we can to watch and protect ourselves, he concluded.

---

Previous Section        Contents         Next Section

---