|
| FDA Home Page | CDRH Home Page | Search | A-Z Index |  |
|
 | ||
  |
| Information
for mammography facility personnel, inspectors, and consumers about the implementation of the Mammography Quality Standards Act of 1992 (MQSA) |
 |
PERFORMANCE EVALUATION OF ACCREDITATION BODIES UNDER THE MAMMOGRAPHY QUALITY STANDARDS ACT OF 1992 as amended by the MAMMOGRAPHY QUALITY STANDARDS REAUTHORIZATION ACT OF 1998 |
 |
Purpose
The Mammography Quality Standards Act (MQSA, the Act) of 1992 (Pub .L. No. 102-539), as amended by the Mammography Quality Standards Reauthorization Acts (MQSRA) of 1998 and 2004 (Pub. L. No. 105-248 and Pub. L. No. 108-365), establishes standards for high quality mammography and requires all facilities to be accredited by a Food and Drug Administration (FDA) approved accreditation body (AB) in order for them to demonstrate that they meet these standards. The FDA may approve either private nonprofit organizations or state agencies to serve as ABs. The MQSA also requires the FDA to submit an annual performance evaluation of the approved ABs to the Senate Committee on Health, Education, Labor and Pension and the House Committee on Energy and Commerce under 42 U.S.C. § 263b(e)(6). This report covers the performance of the ABs under the MQSA from January 1, 2005 through December 31, 2005.
Status of Accreditation Body Approvals
Currently, there are four ABs: the American College of Radiology (ACR), a private nonprofit organization, and the state ABs of Arkansas (SAR), Iowa (SIA), and Texas (STX). The FDA approved each of these ABs under the final MQSA regulations. Since the term of approval is for a period of seven years, each AB’s approval was set to expire on April 28, 2006. Thus, each AB (per regulation) began the renewal process in the fall of 2005. By the end of 2005, the FDA approved each AB’s renewal application. Although the expiration for renewal is April 28, 2013, the FDA will continue to annually review each AB’s performance to determine its compliance with the final regulations.
Standards
Under the MQSA, each AB must require facilities it accredits to meet standards that are substantially the same as the quality standards established by the FDA under 42 U.S.C. 263(f) to assure the safety and accuracy of mammography. All ABs have either adopted the final MQSA standards by reference, or have developed standards that are substantially the same as the quality standards established by the FDA. Each AB incorporated the standards into its own accreditation processes.
Methodology
To assess overall performance, the FDA evaluates the AB’s in the following areas (as outlined in the final MQSA regulations):
The FDA evaluates performance in these areas through:
The FDA staff analyzes unit accreditation pass and fail data, along with data that describe the reasons for each accreditation failure decision. Significant differences in pass and fail rates or reasons for accreditation denial among ABs could, for example, indicate that one AB is interpreting the significance of a particular quality standard more or less strictly than another.
To complement the information submitted by the ABs, the FDA analyzes information from its Mammography Program Reporting and Information System database of annual facility inspections. Accredited facility performance during inspections is measured by average phantom image scores, average radiation dose values, and average processor speeds. Collectively, these measures reflect the overall functioning of all components of the mammography system.
Performance Indicators
(1) Administrative Resources and Funding
AB staffs generally include management, mammography radiologic technologists, MQSA inspectors, health physicists, information technology program application specialists, and administrative assistants. In 2005, all ABs continued to maintain adequate funding and staffing for their respective programs.
(2) Data Management (Process/Errors)
All ABs provide the FDA with electronic transmissions of accreditation data in a secure and appropriately maintained manner. The percentage of data management errors either remained the same or increased slightly (by 5 percent or less) of those noted in the previous year. The FDA continues to work individually with the ABs to
(3) Reporting and Recordkeeping
The FDA’s review of the ABs’ reporting and recordkeeping practices includes examining procedures for handling serious consumer complaints, appeals for accreditation decisions, and granting interim accreditation.
(a) Serious Consumer Complaints
The regulations require ABs to develop and administer a consumer complaint mechanism whereby all facilities that an AB accredits must file serious unresolved complaints with their AB. By regulation, each AB must submit to the agency an annual report summarizing all serious complaints received during the previous calendar year, their resolution status, and any actions taken in response to them.
All ABs have an established appropriate serious consumer complaint mechanism. In calendar year 2005, only two ABs (ACR and SAR) received complaints from a total of five consumers. Each of the ABs submitted its serious consumer complaint report to the FDA which indicated that the ABs followed their approved procedures when resolving these complaints.
(b) Appeals
Each AB must have an appeals process for facilities to contest an AB’s adverse accreditation decision. In calendar year 2005, only two of the ABs received appeals to their accreditation decisions. The ACR received three appeals. It upheld the original adverse decision for one appeal and overturned the other two. The STX received one appeal for which it upheld its original adverse decision.
(c) Interim Accreditation
An AB may grant a 45-day interim accreditation to a fully accredited facility whose MQSA certificate will expire prior to the AB making a renewal decision. The facility must be fully accredited and meet certain criteria in order to obtain interim accreditation. Once the AB grants the facility interim accreditation, the FDA (or state certifying agency) may grant the facility a 45-day interim certificate. Each AB has an interim accreditation policy and procedure in place.
In calendar year 2005, the ACR granted interim accreditation to six of its facilities; the SIA granted interim accreditation to one of its facilities; and the STX granted interim accreditation to six of its facilities. Each AB followed its approved procedure for granting interim accreditation.
(4) Accreditation Review and Decision-Making Processes
Review of the ABs’ accreditation and decision-making processes includes evaluating procedures for clinical image review, phantom image review, and mammography equipment evaluation and medical physicist annual survey review.
(a) Clinical Image Review
As part of the accreditation process, mammography facilities must submit clinical images to their ABs for review. To evaluate the ABs’ performance in the clinical image review area, the FDA’s MQSA qualified interpreting physicians (IPs) annually review clinical images from a sample of facilities that submit cases to the ABs for clinical image review. Generally, two FDA IPs independently conduct clinical image reviews for each facility in the sample for each of the ABs that perform clinical image review, evaluating each examination on the eight attributes listed in the final regulations.
The ACR, the SAR, and the SIA (the STX contracts with the ACR to conduct its clinical image reviews) have their own clinical image reviewers to evaluate their facilities’ clinical images. A summary of the FDA clinical image reviews follows.
American College of Radiology AB
The FDA performed its evaluation of the ACR’s clinical image review process on October 24, 2005. The FDA found that there was good agreement between reviewers at the attribute evaluation level. In reviewing the images and summary evaluation forms, the FDA reviewers agreed with the final overall assessments (pass and fail) in all the cases.
The FDA determined that this spot review of cases indicates that the quality of clinical image review by the ACR remains high and has not deviated from past performance. Overall, the clinical image reviewers are providing adequate feedback to facilities as an educational tool to aid the facilities in improving film quality. However, in several cases, the FDA indicated that the reviewers should have given more constructive criticism regarding inferior mammary folds and the contour of the pectoralis muscle on the medial lateral oblique views.
State of Arkansas AB
The FDA performed its evaluation of the SAR’s clinical image review process in October 2005. The FDA’s IPs agreed with the final overall assessments (pass and fail) in all of the cases reviewed indicating that the quality of clinical image review performed by the SAR remains high and has not deviated from past performance. The FDA IPs observed one issue that required the SAR AB’s attention. On the clinical image review evaluation form, there are two questions that need to be answered, but only in those cases that fail. These questions deal with whether the reviewer believed the exam to be of diagnostic quality and whether an AMR should be considered. Treatment of these questions was quite variable. In some cases that passed, these questions were answered, but in a failed case, they were not. For the failed case, the issue of whether an AMR should have been performed was not specifically addressed by the reviewers on the form. The SAR AB communicated the FDA’s comments in a letter to its clinical image reviewers encouraging them to use the form more appropriately. The FDA will follow-up with the SAR AB during its 2006 evaluation to ensure the forms are being used appropriately.
State of Iowa AB
In October 2005, the FDA performed its evaluation of the SIA’s clinical image review process. The FDA IPs found consistent agreement among the SIA reviewers and agreed with the SIA reviewers’ final overall assessments (pass/fail) in all the cases reviewed. The review indicated that the quality of clinical image review performed by the SIA AB remains high and has not deviated from past performance.
Summary of Audits and Training of Clinical Image Reviewers by the ABs
Audits
An audit of clinical image reviewers ensures uniformity, identifies any potential problems, and provides all individual clinical image reviewers with the necessary data to compare his/her results to the rest of the review group. The ABs use audit results to enhance reviewer training by emphasizing any performance issues. In 2005, the ACR (the STX via its contract with the ACR), the SAR, and the SIA conducted audits of their clinical image reviewers to collect statistics on reviewer agreement and nonagreement rates. For any reviewer that shows poor performance, the AB requires the reviewer to undergo remedial action.
Training
The ACR, the SAR, and the SIA (the STX contracts with the ACR for clinical image review) have clinical image review quality control activities that promote consistency among the various clinical image reviewers. Each of these ABs conducts training sessions at which clinical image reviewers evaluate clinical images and discuss findings, including the application of AB clinical image review evaluation criteria.
(b) Phantom Image Review
As part of the accreditation process, mammography facilities must submit phantom images to their ABs for review. To evaluate the ABs’ performance in the phantom image review area, the FDA’s MQSA expert staff annually reviews phantom images from facilities that submit cases to the ABs for phantom image review. Two FDA staff, working independently, review approximately 10 to 20 randomly selected phantom images from each of the ABs that perform phantom image review. The FDA evaluates all test objects (fibers, specks, masses) on these images as part of the review. Scores for these test objects should fall within the acceptable limit of + 0.5.
The ACR, the SAR, and the SIA (the STX contracts with the ACR to conduct its phantom image reviews) have their own phantom image reviewers to evaluate their facilities’ phantom images. A summary of the FDA phantom image reviews follows.
American College of Radiology AB
The FDA reviewed the ACR’s phantom images on October 24, 2005. The FDA reviewers agreed with the ACR phantom image reviewers on the pass/fail decision in 95 percent of the cases. The FDA determined that this spot review of the phantom images indicates that the quality of phantom image review by the ACR remains high and has not deviated from past performance.
State of Arkansas AB
The FDA reviewed the SAR’s phantom images in November 2005. The FDA compared its scores with the scores of the SAR reviewers and the pass/fail agreement rate between the SAR AB and the FDA was 100 percent. The FDA reviewers indicated that the quality of phantom image review performed by the SAR remains high and has not deviated from past performance.
State of Iowa AB
In November 2005, the FDA reviewed the SIA’s phantom images. The FDA compared its scores with the scores of the SIA reviewers and the pass/fail agreement rate between the SIA AB and the FDA was 100 percent. The FDA reviewers indicated that the quality of phantom image review performed by the SIA remains high and has not deviated from past performance.
Summary of Audits and Training of Phantom Image Reviewers by ABs
Audits
An audit of phantom image reviewers ensures uniformity, identifies any potential problems, and provides all individual phantom image reviewers with the necessary data to compare his/her results to the rest of the review group. The ABs use audit results to enhance reviewer training by emphasizing any performance issues. In 2005, the ACR (the STX via its contract with the ACR), the SAR, and the SIA conducted audits of their phantom image reviewers to collect statistics on reviewer agreement and nonagreement rates. For any reviewer that shows poor performance, the AB requires the reviewer to undergo remedial action.
Training
The ACR, the SAR, and the SIA (the STX contracts with the ACR for phantom image review) have phantom image review quality control activities that promote consistency among the various phantom image reviewers. Each of these ABs conducts training sessions at which phantom image reviewers evaluate phantom images and discuss findings, including the application of AB phantom image review evaluation criteria.
(c) Mammography Equipment Evaluation (MEE) and Medical Physicist Survey Report Reviews
The final regulations state that ABs shall require every facility applying for accreditation to submit an MEE with its initial accreditation application and, prior to accreditation, to submit a medical physicist survey on each mammography unit at the facility (21 CFR 900.4(e)). All of the ABs have established policies and procedures for the review of both the MEE and the medical physicist survey report.
(5) AB Onsite Visits to Facilities
The final MQSA regulations (21 CFR 900.4(f)(1)(i)) require that each AB annually conduct onsite visits to at least five percent of the facilities the body accredits to monitor and assess facility compliance with the standards established by the body for accreditation. However, a minimum of 5 facilities shall be visited, and visits to no more than 50 facilities are required. During such visits, the AB is required to evaluate eight core elements:
At least 50 percent of the facilities visited shall be selected randomly and the other facilities visited shall be selected based on problems identified through state or FDA inspections, serious complaints received from consumers or others, a previous history of noncompliance, or other information in the possession of the AB, the MQSA inspectors, or the FDA (i.e., visits for cause).
American College of Radiology AB
The ACR conducted 47 onsite visits (43 random, four for cause) in calendar year 2005. During the fall of 2005, the ACR scheduled four random onsite visits in the State of Mississippi but these visits were cancelled due to the devastation in the selected area by Hurricane Katrina. Thus, the ACR was not able to complete its required 50 onsite visits. The ACR will include the State of Mississippi in its future onsite visit schedule.
State of Arkansas AB
The SAR conducted 8 onsite visits (6 random, 2 for cause) in calendar year 2005, thus exceeding the minimum of 5 onsite visits required by regulation.
State of Iowa AB
The SIA conducted 48 onsite visits (46 random, two for cause) in calendar year 2005, thus exceeding the minimum of 7 onsite visits required by regulation.
State of Texas AB
The STX conducted 10 onsite visits (7 random, 3 for cause) in calendar year 2005, thus exceeding the minimum of 8 onsite visits required by regulation.
(6) Random Clinical Image Review
The final MQSA regulations (21 CFR 900.4(f)(2)(i)) require that each AB annually conduct RCIRs of at least 3 percent of the facilities the body accredits to monitor and assess facility compliance with the standards established by the body for accreditation.
American College of Radiology AB
During calendar year 2005, the ACR conducted 392 RCIRs, thereby exceeding the 256 required by regulation.
State of Arkansas AB
The SAR conducted 8 RCIRs in calendar year 2005, thus exceeding the minimum of the 2 required by regulation.
State of Iowa AB
The SIA conducted 46 RCIRs in calendar year 2005, thus exceeding the minimum of the 4 required by regulation.
State of Texas AB
The STX conducted 6 RCIRs in calendar year 2005, thus exceeding the minimum of the 5 required by regulation.
(7) Additional Mammography Review
If the FDA has reason to believe that mammography quality at a facility has been compromised and may present a serious risk to human health, the facility must provide clinical images and other relevant information, as specified by the FDA (or Certifying Agency), for review by its AB (21 CFR 900.12(j)). This AMR helps the agency to determine whether there is a need to notify affected patients, their physicians, or the public that the quality of mammograms may have been compromised. The request for an AMR may also be initiated by an AB or a state certifying agency. When an AB initiates an AMR, the FDA encourages it to discuss the case with the agency prior to performing the AMR.
The following chart summarizes the number of AMRs conducted by each AB during calendar year 2005:
AB |
Number of AMRs Conducted or Initiated* |
Number With Deficiency or Serious Risk |
Number That Completed Corrective Action and/or Notification |
|---|---|---|---|
ACR |
19 |
6 |
6 |
SAR |
3 |
1 |
1 |
SIA |
7 |
4 |
2** |
STX |
5 |
3 |
3 |
*Note: The STX has a contract with the ACR to conduct its clinical image reviews during an AMR. The remaining three ABs have their own clinical image reviewers to evaluate their facilities’ clinical images.
**SIA performed two AMRs (limited and full) at one facility and determined there was no serious risk to human health but found deficiencies that required the facility to submit a corrective action plan (CAP). The SIA AB recently met with the facility administrators and staff and it is currently waiting for the facility to submit its CAP.
(8) Accreditation Revocation and Suspension
The MQSA final regulations (21 CFR 900.3(b)(3)(iii)(I)) require that each AB have policies and procedures for suspending or revoking a facility’s accreditation. If a facility cannot correct deficiencies to ensure compliance with the standards or if a facility is unwilling to take corrective actions, the AB shall immediately notify the FDA, and shall suspend or revoke the facility’s accreditation.
Neither the SAR nor the SIA revoked or suspended any facility’s accreditation in 2005.
American College of Radiology AB
The ACR revoked the accreditation of two facilities during 2005. After the ACR performed an AMR on each facility, it issued each a letter of revocation when its clinical image reviewers found the facilities’ practices to possibly pose a serious risk to human health. Subsequently, under 21 CFR 900.13(a), the FDA determined that the certificates at both facilities were no longer in effect and required the facilities to notify affected patients and their referring physicians. One facility took corrective action and ACR is waiting for the facility to submit its documentation in order to be reinstated; the other facility closed.
State of Texas AB
The STX AB did not suspend or revoke the accreditation of any facility in calendar year 2005. However, it did withdraw the accreditation of one facility after the facility voluntarily closed and filed for bankruptcy.
(9) Quantitative Accreditation and Inspection Information
As additional performance indicators, the FDA analyzed quantitative accreditation and inspection information related to unit accreditation pass/fail data; reasons for denial of accreditation; and accredited facility performance during inspections.
Note: There are a relatively small number of state-accredited facilities compared to the ACR-accredited facilities. Therefore, small variations in state-accredited facility performance may lead to differences across ABs that do not reflect actual differences in AB performance.
(a) Unit Accreditation Pass/Fail Data Sorted by AB
Number of Units |
ACR |
SAR |
SIA |
STX |
|---|---|---|---|---|
Total |
4,944 |
24 |
59 |
78 |
Passed Accreditation |
4,933 (99.7%) |
24 (100%) |
59 (100%) |
78 (100%) |
Denied Accreditation* |
11 (0.3%) |
0 |
0 |
0 |
*Units that were still denied accreditation as of December 31, 2005.
At the conclusion of the reporting period, the accreditation pass rate of mammography units among the ABs ranged from 99.7 - 100 percent. The rates for units that were denied accreditation decreased slightly from those in the last reporting period.
(b) Reasons for Mammography Unit Denial
In 2005, clinical image review failure was the major reason for denial of unit accreditation. Phantom image review failure and failure to submit the required materials were the other reasons for mammography units being denied accreditation. Most of the facilities that receive a denial in the accreditation process complete rigorous CAP under the ABs’ reinstatement protocols and eventually successfully achieve the levels of quality needed for accreditation.
(c) Facility Performance During Inspections Sorted by AB
In calendar year 2005, 71.1 percent of the accredited mammography facilities had no MQSA violations. This is an increase from the 2004 report. Also, in calendar year 2005, only 1.9 percent of the facilities had a violation characterized as “most serious.” This is a decrease from the 2004 report. The FDA actively works with these facilities on corrective measures, or takes regulatory measures if a facility cannot improve its performance.
ACR |
SAR |
SIA |
STX |
|
|---|---|---|---|---|
Average Phantom Image Score* |
12.3 |
12.6 |
11.7 |
13.0 |
Average Dose (in millirads) |
176.9 |
173.5 |
159.8 |
181.6 |
Average Processor Speed |
105 |
110.2 |
101.4 |
105.9 |
*The maximum possible phantom image score is 16. Four fibers, three masses, and three speck groups must be visible on the image for a minimum passing score.
There were no significant differences in average phantom image scores among the facilities accredited by the four ABs. In general, average phantom image scores increased slightly from those reported in the 2004 Report.
In general, the average doses remained the same as those reported in the 2004 report and remain well below the dose limit of 300 millirads mandated by the MQSA final regulations. This dose limit has the advantage of permitting flexibility for the optimization of technique factors used during examinations to achieve improved image quality.
The average processing speeds among the facilities of all the ABs decreased slightly from those reported in the 2004 report and remain well within the range to produce satisfactory clinical images. The evaluation of the mammography facility’s film processing speed is an important quality assurance measure. The speed of film processing impacts directly not only on the resulting image quality of the mammogram, but can also impact on the dose administered to the patient. If a mammography facility is processing film in accordance with the film manufacturer’s recommendations, then the processing speed should be close to 100 (80 – 120 is considered normal processing speed for standard cycle processing). If the processing speed falls significantly below the acceptable level, then the clinical image is not completely developed and may appear too light, and the quality of the mammographic image can be significantly compromised. Moreover, the facility may not realize its film processor is the source of the problem and may compensate by increasing the dose administered to the patient.
Status of the Action Items From the 2004 Report to Congress
The one AB with a single calendar year 2004 action item successfully completed its resolution.
Conclusion
The FDA’s AB oversight program promotes collaboration and cooperation. Therefore, each AB, in concert with the FDA, addresses any action items that may arise during the year. The FDA and the ABs, working in partnership with the certified mammography facilities in the United States as well as the states participating in inspection and other MQSA activities are ensuring quality mammography across the nation.
Updated December 1, 2006
CDRH Home Page | CDRH A-Z Index | Contact CDRH | Accessibility | Disclaimer
FDA Home Page | Search FDA Site | FDA A-Z Index | Contact FDA | HHS Home Page
Center for Devices and Radiological Health / CDRH