15
Table 10 shows that, of the uninterrupted inspections, the Northeast had the highest percentage of inspections that ran fewer than four days (72%). Although more of the longer inspections were omitted from the table because they were more often interrupted, the Southeast Regions inspections seem to have run longer than average: 36% lasted more than a week and only a third lasted fewer than four days. Note that none of the foreign inspections lasted longer than a week. The percentage of inspections with interruptions of longer than two working days ranged from none with the foreign inspections to 23% with the Pacific Regions inspections. Fifteen percent of the Southwest Regions respondents reported that their inspections were interrupted; 12% of both the Northeast and Central Regions inspections were interrupted for longer than two days. The domestic region with the smallest percentage of respondents reporting two or more day interruptions was the Southeast Region, with only 6% of its inspections interrupted. The above mentioned interruptions resulted from FDA request (as opposed to firm request or both) roughly two-thirds of the time (60% to 77%) in each district (question 4a). More specific percentages are not provided because the numbers are small and thus the percentages are unstable. Respondents were asked in question 5 whether they were able to have all the right personnel available during inspection. Responses were virtually constant across districts with the Northeast at 94%, Central and Southwest at 93%, foreign at 91%, Southeast at 90%, and Pacific at 89%. Whether or not all necessary records were available to the investigators (question 6) varied a bit across regions. The Southeast Regions respondents reported that they were able to provide all the records needed for the inspection in 97% of the cases. Other regions respondents were able to provide all the records a bit less often: 93% for the Northeast; 91% for Central and foreign; 86% for the Pacific; and 85% for the Southwest. Clearly, the vast majority of all regions respondents were in agreement that they were able to meet the investigators needs. Daily notification of the investigators observations (question 7) was also tabulated by region and is shown in Table 11. Since this question only applies to inspections of more than one day, only such inspections are included in the table below. 16
Table 11. Was the
Firm Always Notified Daily
The foreign firms reported most often that they were always notified daily of the inspectors observations. Three regions, the Northeast, Pacific, and Central, each had about 90% of their responding firms with inspections of more than a day report that they were notified daily of observations. The smallest percentage of reported daily notifications was for the Southeast where 76% reported having been notified daily. Note that since the numbers in Table 11 are a bit smaller than in other tables, the percentages are a bit less stable. The final question examined in this section compares opinions of whether the investigator gave any helpful information or suggestions (question 8) across regions. The foreign firms most often found their investigator to be helpful with 98% reporting that the investigator gave helpful information or suggestions. For the five domestic regions, 96% of the Southeast respondents, 93% of the Central, 90% of the Southwest, and 89% of the Pacific and Northeast respondents reported that the investigator gave helpful information or suggestions.
Outcome of the InspectionThis section contains comparisons across regions of corrective actions taken and promised by the firms and possible problems with their 483s. First, Table 12 shows the firms responses about whether they received an FDA 483 (question 9). 17
Table 12. Issuance of an FDA 483
As shown, roughly 50 to 70% of each regions inspections received 483s. Proportionately more 483s were given in the Southwest and fewer in the Northeast and Central regions, but the differences were not striking. Only the 303 respondents who were issued 483s are included in the remainder of this section. Question 15 asked the firms if all the observations on their FDA 483s were clear. Table 13 shows their responses broken down by region. 18
Table 13. Clarity
of the
The table shows that firms overwhelmingly thought the observations on their 483s were understandable, though the Southwest regions respondents were clear about the observations a bit less often. Respondents were also asked if they found any FDA 483 observations to be inappropriate, as opposed to inaccurate (question 16). Mostly firms felt the 483 observations were appropriate. Differences between the regions were relatively small all were in approximately the 70 to 80 percent range for accuracy (a no response). Foreign respondents most often felt all their 483 observations were appropriate, as 91% of foreign firms gave a no response to the question. Eighty-four percent of the Northeast Regions respondents said that all their FDA 483 observations were appropriate, as did 81% of both the Southwest and Central Regions respondents, 72% of the Southeasts respondents, and 69% of the Pacific Regions respondents. Table 14 shows the percentage of respondents with any errors on their FDA 483s. As in Table 7, Section I, it summarizes responses to questions 14 and 16 so that any respondent who felt that they had a problem on their 483 with a non-annotation related inaccuracy or an inappropriate observation is classified as a yes. Corrective action annotation problems (question 13) and lack of clarity (question 15) were again not considered errors. 19 Table 14. Were
There One or More FDA 483
The most satisfied respondents were from the Foreign and Northeast Regions inspections. The Pacific and Southeast Regions respondents were the least satisfied, with about a third of their respondents reporting at least one non-annotation inaccuracy or inappropriate observation on their 483s. Next we turn to problems with the firms promised and/or taken corrective actions. The vast majority of firms (85% to 98%) in all regions either promised or took corrective actions. Question 11 asked the firms if any of the corrective actions they took could have been verified by the inspector but were not. Again, the vast majority from all regions reported that their investigator had acted appropriately. Ninety-five percent of the foreign respondents whose firms took corrective actions said that the investigator had verified all their corrective actions. For the domestic respondents, 94% of the Northeast Regions respondents who took corrective actions said that all that could have been verified were verified. This figure was 92% for the Southwest Region, 90% for the Southeast Region, 87% for the Pacific Region, and 86% for the Central Region. In other words, there are slightly fewer complaints about investigators not verifying corrective actions in the Northeast, and slightly more complaints in the Central and Pacific regions. However, since this question only pertained to those who responded yes to having taken corrective actions in question 10, the numbers are small and therefore less stable than most of the other percentages given in this report. 20
Question 13 asked if the promised and/or taken corrective actions were properly annotated on the FDA 483. Again, the overwhelming majority of firms reported that they were. For the Northeast Regions respondents, 91% of the firms that took or promised corrective actions reported that all their actions were properly annotated on the 483 and an additional 3% reported that some were properly annotated. For the Southwest Regions respondents, 91% of the firms that took or promised corrective actions reported that all were properly annotated on the 483. No one from the Southwest Region reported partial annotation. Eighty-eight percent of the Central Regions respondents reported full annotation, and an additional 5% reported partial annotation. Similarly, eighty-seven percent of the Pacific Regions respondents reported full annotation, and an additional 6% reported partial annotation. For the Southeast, 79% reported full annotation and another 14% reported partial. Foreign respondents again were among the most content: 88% of those who took or promised corrective actions reported full annotation and 3% reported partial. Although these figures show that respondents criticized the Southeast investigators use of the annotation process a bit more often than those from other regions regarding failure to fully annotate corrective actions on the FDA 483, this difference was small when the instability of the small numbers is taken into account.
After the InspectionThis section compares responses about things that happened at the close of the inspection and afterwards. Seventy-six percent of all 559 firms reported that their highest level executive was present at the final discussion between the investigator and the firms management, the closeout meeting (question 19). The breakdown according to region was: 86% Pacific; 83% foreign; 81% Southeast; 70% Northeast; 69% Central; and 65% Southwest. Of the 303 firms that received FDA 483s, most regions had over 90% of their responding firms report that they planned to respond to the FDA in writing (question 17): 96% of the Southeast firms; 94% of the Central and Pacific firms; 93% of the foreign and Southwest firms; and 84% of the Northeast firms.
Overall Evaluation of InspectionTable 15 shows the respondents opinions about how this inspection compared with previous inspections, for those who had experienced previous inspections. 21 Table 15. How Did
This Inspection Process How Did
This Inspection Process
As shown, the foreign and Central Region respondents were the most positive about this inspection with nearly 60% of each reporting that this inspection was better than previous. The Southwest Regions respondents most often reported that this inspection was worse than previous about 16% said this inspection was worse as compared with about 8% to 10% of respondents in the other four regions. This may be related to the fact that the Southwest Region firms were issued a slightly higher percentage of 483s (Table 12). 22 III. LENGTH OF VS. REASON FOR
|
Reason for Inspection |
1 Day |
2-3 Days |
4-5 Days |
6-10 Days |
More than 10 Days |
Row Total |
|
Preapproval Only | Number | 3 | 10 | 20 | 2 | 35 | |
Row % | 8.6% | 28.6% | 57.1% | 5.7% | 100.0% | ||
QS/GMP Only | Number | 59 | 121 | 109 | 50 | 9 | 348 |
Row % | 17.0% | 34.8% | 31.3% | 14.4% | 2.6% | 100.0% | |
Other Only | Number | 13 | 21 | 15 | 3 | 52 | |
Row % | 25.0% | 40.4% | 28.8% | 5.8% | 100.0% | ||
Preapproval & | Number | 1 | 7 | 7 | 6 | 5 | 26 |
QS/GMP | Row % | 3.8% | 26.9% | 26.9% | 23.1% | 19.2% | 100.0% |
Preapproval & | Number | 1 | 1 | ||||
Other | Row % | 100.0% | 100.0% | ||||
QS/GMP & | Number | 3 | 4 | 9 | 6 | 3 | 25 |
Other | Row % | 12.0% | 16.0% | 36.0% | 24.0% | 12.0% | 100.0% |
TOTAL | Number | 79 | 164 | 160 | 67 | 17 | 487 |
Row % | 49.8% | 33.7% | 32.9% | 13.8% | 3.5% | 100.0% |
23
The table shows that inspections performed solely for preapproval generally lasted 4 to 5 days (the median was 4 days), QS/GMP inspections generally lasted 2 to 5 days (the median was also 4 days), and inspections performed primarily for other reasons were often shorter, generally 2 or 3 days (the median was 3 days).
24
The inspection outcome was provided by the FDA for 306 domestic inspections, about 71% of the domestic inspections covered in this report. Outcomes were classified as NAI, either no FDA 483 or no substantive FDA 483; VAI, substantive FDA 483 but no Warning Letter; or OAI, Warning Letter or worse was received by the firm. Table 17 shows the breakdown of these outcomes for the 306 domestic firms.
Table 17. Current Inspection Outcome
Number | Percent | |
NAI No Substantive 483 | 153 | 50.0 |
VAI Subst. 483, No W/L | 115 | 37.6 |
OAI - W/L or Worse | 38 | 12.4 |
Total | 306 | 100.0 |
In this section, inspection outcome is compared with the number of medical device employees in the firm, preannouncement, length of the inspection, whether the firms highest level executive attended the closeout meeting, and how this inspection compared with previous.
Table 18 shows that there is little relationship between the firms number of medical device employees and the three inspection outcomes except that the largest firms were a bit less likely to be OAI.
25
Table 18. Current
Inspection Outcome
by Firms Number of Employees (Q-20)
Total Number Employed in Medical Device Worldwide |
|||||||
Current Inspection Outcome |
1 to 36 |
37 to 225 |
More than 225 |
Row Total |
|||
NAI - No Substantive 483 | Number | 43 | 54 | 52 | 149 | ||
Column % | 48.9% | 47.0% | 54.7% | 50.0% | |||
VAI - Subst. 483, No W/L | Number | 32 | 42 | 37 | 111 | ||
Column % | 36.4% | 36.5% | 38.9% | 37.2% | |||
OAI - W/L or Worse | Number | 13 | 19 | 6 | 38 | ||
Column % | 14.8% | 16.5% | 6.3% | 12.8% | |||
TOTAL | Number | 88 | 115 | 95 | 298 | ||
Column % | 100.0% | 100.0% | 100.0% | 100.0% |
The majority of the firms inspected, whether they received advance notification or not, were NAI or VAI, as shown in Table 19.
Table 19. Current
Inspection Outcome
by Preannouncement (Q-1)
Did Firm Receive Advance Notification? |
|||||
Current Inspection Outcome |
Yes |
No |
Row Total |
||
NAI No Substantive 483b | Number | 135 | 18 | 153 | |
Column % | 52.1% | 38.3% | 50.0% | ||
VAI Subst. 483, No W/L | Number | 97 | 18 | 115 | |
Column % | 37.5% | 38.3% | 37.6% | ||
OAI W/L or Worse | Number | 27 | 11 | 38 | |
Column % | 10.4% | 23.4% | 12.4% | ||
TOTAL | Number | 259 | 47 | 306 | |
Column % | 100.0% | 100.0% | 100.0% |
The inspection outcome and length of the inspection were related. Table 20 below shows results consistent with those in Section III of this report: the more serious the problems found the longer the inspection. This table shows the 267 firms for which both outcome results were available and length of inspection could be estimated.
26
Table 20. Current
Inspection Outcome
by Length of Inspection
(if Not Interrupted by More than 2 Days)
Length
of Inspection (if Not Interrupted by More than 2 Days) |
||||||||
Current Inspection Outcome |
1 - 3 Days |
4 - 5 Days |
6 - 10 Days |
More Than 10 Days |
Row Total |
|||
NAI - No Substantive 483 | Number | 98 | 32 | 14 | 2 | 146 | ||
Column % | 71.0% | 47.1% | 28.0% | 18.2% | 54.7% | |||
VAI - Subst. 483, No W/L | Number | 35 | 28 | 23 | 6 | 92 | ||
Column % | 25.4% | 41.2% | 46.0% | 54.5% | 34.5% | |||
OAI - W/L or Worse | Number | 5 | 8 | 13 | 3 | 29 | ||
Column % | 3.6% | 11.8% | 26.0% | 27.3% | 10.9% | |||
TOTAL | Number | 138 | 68 | 50 | 11 | 267 | ||
Column % | 100.0% | 100.0% | 100.0% | 100.0% | 100.0% |
Finally, the inspection outcome was compared with whether the respondent felt this inspection was better or worse than previous inspections. Table 21 shows that there is a weak relationship between how much the respondent liked this inspection and how favorable its outcome.
Table 21. Current
Inspection Outcome by
How Did This Inspection Process Compare
with Previous (Q-18)?
How
Did This Inspection Process Compare with Previous? |
|||||
Current Inspection Outcome |
Better |
Same |
Worse |
Row Total |
|
NAI - No Substantive 483 | Number | 63 | 66 | 2 | 131 |
Column % | 50.4% | 58.4% | 8.7% | 50.2% | |
VAI - Subst. 483, No W/L | Number | 50 | 39 | 12 | 101 |
Column % | 40.0% | 34.5% | 52.2% | 38.7% | |
OAI - W/L or Worse | Number | 12 | 8 | 9 | 29 |
Column % | 9.6% | 7.1% | 39.1% | 11.1% | |
TOTAL | Number | 125 | 113 | 23 | 261 |
Column % | 100.0% | 100.0% | 100.0% | 100.0% |
It is interesting to note that 12 of the OAI respondents felt this inspection was better than previous and two of the NAI respondents felt this inspection was worse than previous. Neither of those two reported having received an FDA 483. One said that this inspection was worse because it took more time, the other said it was worse because employees had to cancel vacations to accommodate the inspection schedule.
27
For the 12 respondents with very unfavorable inspection results who were still very positive about the inspection, four cited the positive attitude of their investigator (understanding, helpful, courteous, flexible, knowledgeable), two liked the comprehensive nature of the inspection, one liked that it was shorter and more focused, one mentioned QSITs predictability, two mentioned good communication and daily wrapups, and one of those two also mentioned preannouncement.
28
Task Force Subcommittee Members:
Nancy Singer, AdvaMed and Denise Dion, FDA
Lauren Andersen, AdvaMed & Andersen Caledonia Ltd.
Elaine Messa, Quintiles Consulting & Former Director of Los Angeles District Office,
FDA
Leif Olsen, AMDM & BioWhittaker
Susan Reilly, ASQ Biomedical Division & Reilly and Associates
Participating FDA Officials: | Participating Industry Officials: | |
Ronald G. Chesemore Former ACRA, ORA |
Lauren Andersen AdvaMed & Andersen Caledonia Ltd. |
|
Bruce B. Burlington, M.D. Former Director, CDRH |
Wayne Barlow MDMA & Wescor |
|
Deborah D. Ralston Director, ORO |
Thomas Henteleff CLI & Kleinfeld, Kaplan and Becker |
|
Lillian Gill Director, OC, CDRH |
Ernest S. Malachowski CMDA |
|
Gary G. Dean Former Director, DEN-DO |
Thomas Meskan Medical Alley |
|
Edward Esparza Former RFDD, SWR, ORA |
Elaine Messa Quintiles Consulting & Former Dir. LOS-DO |
|
Denise Dion Investigator, ORA |
Leif Olsen AMDM & BioWhittaker |
|
Susan Reilly ASQ Biomedical Div. & Reilly and Assoc. |
||
Nancy Singer AdvaMed |
The Medical Device Industry Initiatives Grassroots Task Force wishes to thank the FDA officials who coordinated the distribution of the Medical Device Inspection Evaluation, and the industry officials who took the time to fill out the questionnaire and return it to the University of California, Irvine, Center for Statistical Consulting.
We feel that this survey has been valuable in that it (1) provided firms an opportunity to give anonymous feedback to the FDA and to industry about their inspection experience; (2) allowed comparisons across regions of companies' reactions to inspections; and (3) helped determine if the medical device industry initiatives (pre-announced inspections, annotated 483s, etc.) were being followed.
29
Feedback to the FDA consisted of (1) a quantitative analysis of the survey results and (2) the many comments which respondents wrote on the questionnaires. The FDA Office of Regulatory Affairs and Center for Devices and Radiological Health management have received a thorough report of the analyses of the questionnaire data. They have also been provided with all the many comments respondents wrote, both short comments written in response to specific questions and longer comments written at the end of the questionnaire. Before forwarding them, comments were stripped of any specifics which might possibly have allowed identification of the company, including FDA district and region, dates of inspection, product being inspected, inspection outcome, and anything unique about the inspection, etc. Comments were typed and categorized according to content by the UC Irvine Center for Statistical Consulting before being forwarded to the FDA.
Regional differences appear to be minimal, but the Office of Regulatory Affairs is continuing its ongoing efforts to assure uniformity and consistency in inspections and enforcement.
In light of the fact that 52% of firms believed the inspection was better than previous inspections, the Committee believes that the medical device industry initiatives of pre-announcing inspections and annotating Form FDA 483s are causing the medical device industry to view the inspection process in a more positive light than it has in the past. The Committee was pleased that most inspections are pre-announced, and 58% of the domestic and 99% of the foreign companies were given five or more days of advance notice before the start of an inspection.
The actual questionnaires have now been shredded by the UC Irvine Center for Statistical Consulting. Only the electronic data file remains, and it has been stripped of all fields that might allow identification of respondents, including region, district, dates of inspection, and all comments.
30
The FDA investigator gave the questionnaire (Appendix B), a reply envelope addressed to Dr. Anita Iannucci at UC Irvine, and the following cover letter to the firms representative at the close of inspection. The cover letter was printed on UC Irvine stationary white paper with blue and black ink. All three items were together in a UC Irvine stationary envelope without any addressee, but with a UC Irvine return address.
31
Questionnaire Cover Letter (PDF 79KB)
32
The questionnaire was printed in the form of a booklet, two double-sided sheets of blue ink on white paper. The FDA investigator was to have filled out the box at the top of page 1 before giving the questionnaire packet to the firm, but in actual practice this was not always done. When the investigator left the box empty, often the firm filled in the information. When it was returned blank, my assistant telephoned the firm to obtain the information. We did not contact the FDA for the data because the FDA was not to know which firms returned their questionnaires and which did not.
33
Medical Device Inspection Evaluation Questionnaire
(PDF 148KB)
34