Comments on the Draft FSIS Listeria Risk Assessment

On February 26, 2003, FSIS held a public meeting to present the draft FSIS Listeria risk assessment, including the model, data, and underlying assumptions. At this meeting, the Agency announced that it would like to receive additional public input and information through Docket 03-005N. The comment period closed on March 14, 2003. FSIS received several substantive comments on the FSIS Listeria risk assessment from both industry and consumer groups (addressed below). Other comments included editorial changes to the draft risk assessment report, which will also be used to finalize this risk assessment report, but do not change the model or its outputs.

The FSIS Listeria risk assessment is comprised of two primary components: 1) a dynamic in-plant Monte Carlo model (referred to as the in-plant model); and 2) the FDA/FSIS exposure assessment pathway for deli meats and the dose-response relationship for L. monocytogenes. The in-plant model quantitatively characterizes the relationship between Listeria species in the in-plant environment and L. monocytogenes in ready-to-eat product at retail was developed using currently available data. The outputs of the in-plant model (e.g., outputs relating to concentration of L. monocytogenes on deli meat at retail) have been used as inputs into specific components of the updated FDA/FSIS risk ranking model. The outputs of the in-plant model were calibrated to the concentration of L. monocytogenes in ready-to-eat product at retail in the updated FDA/FSIS exposure assessment retail to table pathway for deli meats. The FDA/FSIS exposure assessment then tracks the level of L. monocytogenes in ready-to-eat product (i.e., deli meat) from retail to table, and provides estimates of the subsequent risk of illness or death from consuming these ready-to-eat products. Overall, most stakeholder comments focused on the data and assumptions underlying the in-plant model.

Comment: The FSIS Listeria risk assessment did not fully consider all the pathways of contamination of ready-to-eat product after lethality treatment, specifically direct deposition from air and via non-food contact sources. Instead, they said, the FSIS Listeria risk assessment only considered contamination that comes from a reservoir (a niche, or harborage site). Some commenters noted that whether or not the contamination is from a niche or other similar harborage point, or from a transient source, the contamination has the potential to directly contaminate food contact surfaces or product.

Response: While deposition from air can occur, published data to date suggest that it is a limited route of contamination. Tompkin (2002) stated: “A rather common misconception is that air is a notable source of contamination. Throughout 14 years of investigation (unpublished data), the air in a room has never been found to be a chronic source of contamination of product contact surfaces.”

Comment: An FSIS in-depth verification of a single plant was used to develop a non-peer reviewed report with data used to estimate the frequency of a contamination event. Such data were not publicly available and may not be representative of other establishments. Further, the quality of these data must be questionable since the data were not also used to estimate the duration of a contamination event.

Response: The in-depth verification data were considered during the evaluation of the time between contamination events, but the duration of the sampled period was too short to estimate this parameter.

Comment: The Agency should use its own plant data rather than the Tompkin (2002) data used to estimate the duration of a contamination event.

Response: Tompkin’s (2002) data were peer reviewed, represented industry data, and were likely more representative than targeted environmental sampling data.

c. Level of Listeria species transferred from the Plant Environment to Food Contact Surface

Response: Calibration has been used for decades as a standard step in the modeling process, particularly when specific parameter values are unknown and relevant data exist. Note that model calibration is distinct from model validation. For references, see:

Anderson, M.P. and W.W. Woessner. Applied Groundwater Modeling. Academic Press, San Diego,1992.

Beck, J.V. and K.J. Arnold. Parameter Estimation in Engineering and Science. John Wiley & Sons, New York, 1977.

Kleijnen, J. and W. van Groenendaal. Simulation: A Statistical Perspective. John Wiley & Sons, Chichester, 1992.

National Research Council. Ground Water Models: Scientific and Regulatory Applications. National Academy Press. 1990.

Orlob, G.T. (ed.). Mathematical Modeling of Water Quality: Streams, Lakes, and Reservoirs. International Series on Applied Systems Analysis. John Wiley & Sons, Chichester, 1983.

Schwartzman, G.L. and S.P. Kaluzny. Ecological Simulation Primer. Macmillan, New York, 1987.

Comment: Without subtyping data (e.g., PFGE, ribotyping), it is questionable whether the same Listeria strain was isolated in all situations and whether the data used in estimating parameters for a contamination event are very representative.

Response: FSIS believes that were an establishment to find Listeria spp. on a food contact surface, that finding would be indicative of a sanitation problem that could cause potential adulteration of the product (e.g., cross-contamination). The Agency agrees that additional data on the ecology and transfer of Listeria specified by strain would be useful in future updates of this risk assessment. Research of this nature has been conducted for establishments producing dairy or seafood product (i.e., Norton DM et al., Molecular Studies on the Ecology of Listeria monocytogenes in the Smoked Fish Processing Industry, Applied and Environmental Microbiology 67(1):198-205, 2001; Hoffman AD et al., Listeria monocytogenes Contamination patterns for the Smoked Fish Processing Environments and for Raw Fish, J. Food Protection 66(1):52-60, 2003; and Wiedmann M, Molecular Subtyping Methods for Listeria monocytogenes, J. AOAC Int 85(2):524-531, 2002). Similar research on the ecology of L. monocytogenes in ready-to-eat meat and poultry processing establishments would provide additional detailed information.

Comment: The efficacy of sanitation is higher than the value used in the FSIS Listeria risk assessment model. Instead of an efficacy of 75% and 95% reduction in Listeria species on food contact surface during daily and enhanced sanitation, respectively, industry commented that the efficacy should be: (1) 99% for daily sanitation; and (2) 100% for enhanced sanitation.

Response: Clean-up effectiveness measures the proportion of bacteria on the food-contact surface that is removed through sanitation procedures. The model assumes the effectiveness of clean-up between lots is 50% and end of day clean-up is 75%. Therefore, total effectiveness of routine cleaning is actually 1-[(1-50%) H (1-75%)]=87.5%, or just less than a one log₁₀ reduction in the amount of contamination remaining on food contact surfaces. A similar level of effectiveness was estimated for cleaning of stainless steel surfaces experimentally inoculated with a biofilm of Pseudomonas aeruginosa and Staphylococcus aureus (Gibson H, Taylor JH, Hall KE, and Holah JT. Effectiveness of cleaning techniques used in the food industry in terms of the removal of bacteria biofilms. J. Applied Microbiology 87:41-48, 1999). While some plants may achieve greater log reductions from their cleaning practices, the effectiveness levels assumed in this risk assessment seem reasonable as averages across the entire industry.

Regarding enhanced cleaning, it seems unreasonable to assume an infinite log reduction. Such a level of effectiveness could never be proven experimentally. Nevertheless, our analysis of these inputs suggests the model is insensitive to higher effectiveness levels because much of the contamination on food contact surfaces is transferred to RTE deli meats during the time of processing.

Comment: The food contact surface area sampled varies and, for large surface areas, can range from 20-200 square inches per sample site.

Response: The Agency conducted a sensitivity analysis of the sampled food contact surface area and found the model results to be highly insensitive to changes in this parameter. These results were presented in the risk assessment. Furthermore, the baseline model assumes a stochastic food contact surface area tested, allowing for variability from lot to lot of ready-to-eat product.

Comment: The assumption that plant size affects food contact surface area is much less relevant than other factors such as process line configuration, Listeria control program implementation, and packaging technology.

Response: Based on the FSIS RTE Survey, the average mass of a lot of ready-to-eat product varied by plant size. Furthermore, there is no evidence of a difference in the occurrence of L. monocytogenes in ready-to-eat product by plant size. To reconcile differences in lot mass with equivalency in L. monocytogenes occurrence by plant size, an adjustment is made in the model to food contact surface sizes. This adjustment eliminated the unintended bias that would occur by assuming the same food contact surface size regardless of plant size.

No survey data of plant characteristics (e.g., line configuration, Listeria control program implementation, and packaging technology) or corresponding data on the prevalence and/or level of Listeria species in the establishment were provided to the Agency. Therefore, these factors cannot be further evaluated at this time.

Response: The Agency ran a sensitivity analysis on both the food contact surface area sampled and the mass of product sampled. The resulting concentration distribution and public health impacts are equivalent to composite sampling. These results are included in the risk assessment report.

Comment: Several comments focused on the issue of the likelihood of detecting a L. monocytogenes positive lot of ready-to-eat product given finding a food contact surface positive for Listeria species. Some comments suggested that the analysis was not correctly done and to simply consider International Criteria for Microbiological Safety of Food binomial tables. Other comments thought that the limit of detection would be significantly less than what was used in the risk assessment model.

Response: The Agency believes that these comments indicate a consistent misinterpretation of the analysis and will further clarify. For both contact surface testing and product testing, the modeled concentration was multiplied by the sample size to estimate the mean of a Poisson distribution. (For food contact surfaces, the concentration is measured in cfu/cm² and the sample size is measured in cm². For RTE product, the sample size is measured in cfu/gram, and the sample size in grams.) A random number was generated from this distribution which represented the number of cfu’s in the sample itself.

Once the number of organisms in the sample was known, the positive or negative test result could be determined based on a binomial distribution. If p is the probability of detecting 1 cfu in the sample, then the probability of finding the sample positive is

where n is the number of cfu’s in the sample from the Poisson calculation. The p probability is based on the detection limit and microbiological test sensitivity, and is the input parameter to the risk assessment model.

As for the limit of detection, this input was based on the FSIS Microbiological Lab Book (http://www.fsis.usda.gov/OPHS/microlab/mlg8.03.pdf), which reports the detection limit for L. monocytogenes testing as better than 1 cfu in a 25-gram sample. Thus, the p value should be fairly high for L. monocytogenes testing, conceptually near 1, because the base data set assumed a 25 gram sample. Moreover, Hayes et al. (1992) reported that the USDA method for L. monocytogenes had an overall sensitivity of 74%, with a sensitivity of 75% for the luncheon meat subcategory [Hayes, P.S. et al. Comparison of Three Selective Enrichment Methods for the Isolation of Listeria monocytogenes from Naturally Contaminated Foods. J. Food Protection 55(12): 952-959, 1992.]

Comment: Some commenters wondered why the FSIS RTE survey data were not used to estimate the number of shifts per day and the number of operation days per month rather than assume there are two shifts per day and 30 days per month.

Response: The FSIS RTE survey was used along with input from industry to garner information and data on the number of shifts per day and estimated days of operation monthly.

E. Transfer of Listeria species from the Food Contact Surface to RTE Product

Comment: The transfer of Listeria species from food contact surfaces to ready-to-eat product may vary by product configuration (e.g., stacked, shingled), surface physical characteristics, and general operational parameters.

Response: Given the limited evidence currently available regarding transfer of Listeria species from food contact surfaces to ready-to-eat product, the level of detail in the model seems appropriate. Complicating the model to consider product configuration, surface characteristics or other factors would seemingly require additional scientific evidence.

Comment: The data used to model the transfer of Listeria species from food contact surface to ready-to-eat product were not appropriate because the transfer coefficient was an average of data in the literature for different product surfaces, or involved data from an experiment using raw beef rather than ready-to-eat product, and/or was based on data suggestive of a retail rather than in-plant setting. Some commenters recommended using data from Lunden, Autio and Hannu (Transfer of Listeria species associated with a dicing machine. J. Food Protection 65(7):1129-1133, 2002) and from a study on transfer sponsored by industry and conducted by the University of Georgia.

Response: Because the food contact surface was modeled as a single value, an average transfer coefficients across different surface media was deemed appropriate.

The University of Georgia study was evaluated for the risk assessment, but little useful data could be obtained. There are two reasons for this. The first is that the study was conducted at the package level, not a lot level as used in the risk assessment. The second and more important reason is that only prevalence was examined, making it impossible to calculate a transfer coefficient.

The following examples illustrate this point. They are based on Day 1 25 gram sampling for Trial 2, but similar examples could be constructed for any of the results. The slicer was inoculated with 1080 cfu L. monocytogenes. Ten of the 100 samples tested positive for Lm. The table below presents 3 possible scenarios consistent with the data, assuming that 10 cfu transferred to the package would be sufficient to find the sample positive. (This number is probably higher than needed, but only impacts the minimum transfer coefficient calculated.)

In Case A, the minimum number of cfu is transferred to each sample. The vast majority of the cfu’s remain on the slicer, for an overall transfer coefficient of 0.09. In Case B, all the cfu’s are transferred to the samples, leaving none on the slicer and resulting in a transfer coefficient of 1.0.

Package #	Case A		Case B
Package #	Lm Slicer	Lm Package	Lm Slicer	Lm Package
Inoculum	1080		1080
1	1070	10	90	990
3	1060	10	80	10
5	1050	10	70	10
7	1040	10	60	10
9	1030	10	50	10
39	1020	10	40	10
117	1010	10	30	10
195	1000	10	20	10
197	990	10	10	10
199	980	10	0	10
Transfer Coef	0.09		1.00

The observed data are consistent with a transfer coefficient that ranges from 0.09 to 1.00. Prevalence data cannot be used to impute a transfer coefficient. Because of this range, the study was not used directly in the risk assessment, especially since a relevant quantitative study was available in the peer-reviewed literature.

A prevalence of 0 can still imply a non-zero transfer coefficient if the number of organisms transferred to each package is below the detection limit. A prevalence of 100% can still imply a transfer coefficient near 1 if only a small number of organisms are transferred to each package. Thus the University of Georgia study had little relevance to the risk assessment. Because the planned Lm quantitation was unable to be performed, the ARS frankfurter study had similar limitations.

As for the Lunden et al article, this study considered sequential L. monocytogenes contamination at 3 plants as a dicing machine was moved from plant to plant. As such, it speaks more to the fact that food processing equipment can act as long-term harborage sites than to a value for transfer coefficients. Long-term harborage sites are consistent with the conceptual model presented and used in the FSIS Listeria risk assessment.

Comment: The assumption that ratio of the level of Listeria species to L. monocytogenes would be similar to the ratio of the prevalence needs to be validated.

Response: Given the lack of specific data, the assumption that the ratio of L. monocytogenes to Listeria species prevalence applies to the ratio of the concentrations is a reasonable use of available data. Moreover, in a review of this risk assessment, it was found that the assumed truncated normal (52%, 26%) distribution compared to a non-parametric empirical cumulative distribution of the data provides a reasonable fit

Comment: The efficacy of post-processing interventions such as high pressure processing and post-packaging heat treatments is higher than the values used in the FSIS Listeria risk assessment model. Instead of an efficacy of 90-95% reduction in L. monocytogenes in ready-to-eat product, the model should reflect close to 100% effectiveness for this parameter.

Response: The default assumptions regarding efficacy of post-processing interventions used in the model may very well be lower than efficacies observed in plants or laboratories. Simulating a higher efficacy will illustrate greater benefits for these interventions. The current model settings, therefore, are conservative. For example, the current model predicts that post-processing interventions are at least as effective as a testing program that tests every lot of product. Therefore, the model already informs decision-makers that post-processing interventions that are 90%-95% efficacious are as effective, or more effective, than testing.

Comment: The FSIS Listeria risk assessment did not adequately consider the effect of ready-to-eat meat and poultry products that do not support the growth of L. monocytogenes as a result of reduced pH, water activity, or frozen storage, in addition to inhibitors.

Response: The risk assessment modeled the behavior of the deli meat category consistent with the FDA/FSIS risk assessment model. Further consideration of food characteristics (e.g., pH, water activity, presence of growth inhibitors) is already considered in the development of the rule

Comment: The data on interventions, such as the use of lactate and diacetate to prevent growth during distribution, which has been published (Seman et al.. Modeling the growth of Listeria. monocytogenes in cured ready-to-eat processed meat products by manipulation of sodium chloride, sodium diacetate, potassium lactate and product moisture content. J. Food Protection 65:651-658, 2002) were not used.

Response: These data were reviewed during the development of the risk assessment. However given the nature of the management questions, which did not deal with specific product formulations, it was decided to model growth inhibition in a manner which could easily be applied to any product reformulation or packaging. Moreover, as mentioned previously in the context of post-processing controls, the efficacy of growth inhibitors assumed in the model may be conservative. Nevertheless, decision-makers can determine from this model’s results that growth inhibitors are as effective, or more effective, than testing food contact surfaces. Simulating higher efficacy from growth inhibitors only serves to reinforce this determination. In addition, greater percent reductions were modeled as part of the sensitivity analysis and did show greater public health impacts.

Comment: On the adjustment of the growth of L. monocytogenes in ready-to-eat product as it moves from plant to retail, some commenters thought the use of a multiplier of one log was an oversimplification. Others said that such this growth multiplier is likely to low and would result in an underestimate of the risk of illness from L. monocytogenes.

Response: As discussed in the risk assessment, there appear to be contradictions between the reported prevalence of L. monocytogenes in ready-to-eat product produced at the plant compared to recent data on the prevalence found at retail. This approach was considered to be the most consistent with the FDA-FSIS model approach (use of a multiplier) and most appropriate given the conflicting data (selection of 1 log as multiplier) (see Appendix B of the FSIS Listeria Risk Assessment report).

Comments on the Retail-to-Table Portion of the FSIS Listeria Risk Assessment

Comment: The American Meat Institute re-submitted comments initially submitted in 2001 in response to the draft FDA/FSIS Listeria risk ranking model. These comments included the availability of consumer behavior data and focused on the need for using risk assessment models to consider strategies to mitigate the risk of exposure to L. monocytogenes.

Response: The consumer behavior data submitted in 2001 were used to update the exposure pathway for storage and handling of frankfurters in the FDA/FSIS risk ranking model. Secondly, the FSIS Listeria risk assessment was developed in response to risk management questions specifically to evaluate the effectiveness of these interventions to mitigate the public health risk of illness from L. monocytogenes in ready-to-eat food.

Comment: The National Food Processors Association (NFPA) submitted galley proofs of two articles that have since appeared in the Journal of Food Protection. One article relates to survey data of the prevalence and level of L. monocytogenes in ready-to-eat foods at retail (Gombas et al., Survey of Listeria monocytogenes in ready-to-eat foods, Journal of Food Protection 66(4): 559-569, 2003). The other discusses a retail-to-table risk assessment using the recent NFPA retail survey data of L. monocytogenes in ready-to-eat foods (Chen et al., Listeria monocytogenes: Low levels equal low risk, Journal of Food Protection 66(4): 570-577, 2003).

Response: The NFPA data, along with other published data, were used to update the information on the prevalence and level of L. monocytogenes in the FDA/FSIS exposure pathway for deli meats at retail.

Comment: The risk assessment erroneously assumes that all L. monocytogenes on ready-to-eat product at retail arose from contamination at the manufacturing plant. Gombas, et al. (2003) have demonstrated that this assumption is incorrect.

Response: Consideration of contamination occurrences beyond the processing facility was beyond the scope of the model. Conclusions drawn from the in-plant model regarding efficacy of alternative interventions, however, would seem to be unaffected by this consideration. Furthermore, adjustments to benefits resulting from interventions at the processing level, and estimated using the current model, can be considered in the Agency’s analysis of the final regulation.

A. Generalizability of FSIS Listeria Risk Assessment for Deli Meats to other RTE Meat and Poultry Products

a. Comment: Some commenters cautioned that extending extrapolations from this risk assessment for deli meats to other READY-TO-EAT products.

b. Response: The FSIS Listeria risk assessment was developed to inform specific risk management questions using the available scientific information. Given: 1) the availability of data on deli meats; and 2) the fact that deli meats account for most of the annual cases of listeriosis from ready-to-eat foods (based on the FDA/FSIS risk ranking model), the focus of this risk assessment is considered reasonable (i.e., the Agency has chosen to focus its risk assessment on a ready-to-eat product that poses the greatest per annum risk based on the FDA/FSIS risk ranking model (i.e., deli meats) for which data are available). The impacts observed with other ready-to-eat product would follow a similar pattern.

Comment: Since the FDA/FSIS risk-ranking model will not be released until this summer, the FSIS Listeria risk assessment (which uses the exposure pathway for deli meats and dose-response relationship from a revision of this model) is not transparent.

Response: The exposure assessment pathway for deli meats and dose-response relationship is from the draft FDA/FSIS risk-ranking model that has been updated based on public comments. Changes to the exposure pathway for deli meats are included in Appendix A of the risk assessment report. Moreover, data used to make these updates to the exposure assessment pathway for deli meats from the 2001 FDA/FSIS risk-ranking model (posted on the web at: http://www.foodsafety.gov/~dms/lmrisk.html) are available in Docket 03-005N.

Comment: Several comments focused on the need for the FSIS Listeria risk assessment model to be made available to the public to better understand the risk assessment, provide additional transparency, and to further evaluate the model.

Response: FSIS plans to make its Listeria risk assessment model available. The source code for the in-plant model has already been posted on the Agency’s Web site; anyone wishing to further evaluate the details of this component of the risk assessment model is able to do so.

Comment: A few comments suggested that the FSIS Listeria risk assessment receive a peer review prior to it use in informing decision-making.

Response: The Agency has had the FSIS Listeria risk assessment peer reviewed. External reviewers included risk assessment modelers familiar with dynamic and process models, the microbiology of Listeria, and the public health issues associated with L. monocytogenes. All reviewers commended the Agency for producing this type of risk assessment in a limited timeframe in order to provide a scientific basis for food safety decision-making. Overall, the peer review indicated that the FSIS Listeria risk assessment was appropriate to inform decision-making with regards to the specific risk management questions under consideration.

Comment: Several commenters suggested that additional data be used in the FSIS Listeria risk assessment model.

Response: The FSIS Listeria risk assessment was based on currently available data in the peer review literature or provided to the Agency. In some instances, presentations were made to the Agency, but none or limited data were made available to the Agency. Data used in the risk assessment must be made available in the docket so that the risk assessment is transparent and reproducible. Anecdotal evidence and statements made about experience need to be supported by data to provide a sound scientific-basis for risk assessments. The Agency welcomes the submission of data to the docket for consideration.

Comment: The FSIS Listeria risk assessment model should include additional detail, including modeling various types of food contact surfaces, additional operational steps based on the type of ready-to-eat product, additional interventions, and pathways of contamination of food contact surface or product from the plant environment.

Response: The current model was designed specifically to answer the risk management questions posed by Agency risk managers. The current level of detail in the FSIS Listeria risk assessment is adequate to inform decision-making based on these risk management questions. To incorporate additional operational steps and variability in the FSIS Listeria risk assessment model requires the availability of additional data adequate to provide this level of detail. Such data have not been made available to the Agency.

REGULATIONS AND DIRECTIVES DEVELOPMENT

**Comments on the Draft FSIS Listeria Risk Assessment**

Send mail to webmaster with questions or comments about this web site.
Last modified: June 11, 2003

REGULATIONS AND DIRECTIVES DEVELOPMENT

Comments on the Draft FSIS Listeria Risk Assessment

Send mail to webmaster with questions or comments about this web site. Last modified: June 11, 2003

**Comments on the Draft FSIS Listeria Risk Assessment**

Send mail to webmaster with questions or comments about this web site.
Last modified: June 11, 2003