(Appendices 1-4 available in PDF)
| Topic Areas | Public Comment: 2001 Draft Risk Assessment | FDA/FSIS's Response |
|---|---|---|
| Assumptions | Five factors (i.e., amount and frequency of consumption of the food; frequency and levels of Listeria monocytogenes in ready-to-eat food; potential to support growth of Listeria monocytogenes in food during refrigerated storage; refrigerated storage temperature; and duration of refrigerated storage before consumption) affecting consumer exposure to Listeria monocytogenes at consumption are not necessarily additive or equally relevant. | The statement about five factors that influence exposure is an interpretation of the results of the risk assessment. Further work provided in the 2003 risk assessment ('what if' scenarios) gives examples of how factors such as storage time and temperature interact to influence risk. (See Chapter VI. 'What If' Scenarios.) Any of these factors can affect potential exposure to Listeria monocytogenes from a food category. These factors are 'additive' in the sense that when more than one of these factors favor a higher level of Listeria monocytogenes, the foods are more likely to have an increased consumers' risk of listeriosis than when only one factor is high. |
| Assumptions | This risk assessment doesn't consider contamination in homes, daycare centers, schools and other non-retail places. One cannot assume that they don't need to be incorporated into this risk assessment. Further, it is unacceptable to assume that such data do not need to be included just because no such data are available. | A consideration of sources of contamination from homes, daycare centers, schools, and other non-retail establishments is beyond the scope of this risk assessment. If data become available, these sources could be included in future risk assessment projects. The likely impact of these sources of contamination on the predicted risks is not known, however, the epidemiology of outbreaks and sporadic cases suggests that a majority of cases are associated with initial contamination prior to the home. |
| Assumptions | Inherent characteristics and processing methods of foods that result in Listeria monocytogenes inhibition are not taken into account. | A consideration of processing methods was outside the scope of this risk ranking approach. |
| Assumptions | Assumptions should be more protective of public health. | The risk assessment was purposely designed to minimize bias, focusing on the most accurate assessment of risk and its associated uncertainty that can be derived from the available scientific information. This "bias neutral" approach is critical for transparency and appropriately places the decision about the degree of precaution required to deal with scientific uncertainty with risk managers. |
| Assumptions | Multiple speculative assumptions regarding storage conditions and dose response models do not constitute a valid scientific basis for conclusion that Listeria monocytogenes is a risk in retail establishments. | Newly available data on consumer handling of frankfurters and deli meats were incorporated into this risk assessment. The epidemiological records, outbreak and recall data indicate that many of these foods do pose a risk. The dose response models are anchored to the CDC surveillance data. The models developed and conclusions reached were based on the best available scientific data and expert judgment. |
| Assumptions | Some aspects of the exposure assessment contribute to the mischaracterization or over-estimation of risk associated with specific food categories. | The specific food categories were reviewed and discussed with subject matter experts and advisory committees to ensure that assumptions and modeling approach used were consistent with the unique characteristics of foods. |
| Assumptions | Deli salads are not known to have directly caused listeriosis, but the risk ranking places them above products that have (i.e., frankfurters, pasteurized milk, soft mold-ripened and blue-veined cheese, etc). This relates to an assumption in growth rate (use of deli meats as surrogate). | New data became available and the assumed values for growth rates were replaced with data specific for this food category. (See Chapter III. Exposure Assessment.) |
| Assumptions | This risk assessment reports no listeriosis cases resulting from deli salads or frankfurters. Why were they included? Also, their risk is over-estimated as a result of assumptions in exposure assessment. | Deli-type Salads food category was revised with newly available data. Foods were included because of associations with outbreaks, recalls, and availability of contamination data. The epidemiology of cases associated with frankfurters is discussed within the technical document. (See Chapter II. Hazard Identification.) |
| Assumptions, Distribution | Justify changing the weight of the BetaPert from 4 to 7. | The distribution is based on expert judgment, after examining proposed shape of the curve. The standard BetaPert had too many servings stored for long periods of time. |
| Assumptions, Growth | The potential to support growth should be a primary risk factor; refrigeration temperature and storage time should be sub-points, since many foods don't support growth. If the micro-organism cannot grow, temperature and time are not relevant to illness. | The 2003 risk assessment includes scenario testing to evaluate the impact of refrigerator temperature and storage time on the predicted risk. (See Chapter VI. 'What If' Scenarios.) These 'what if' scenarios indicate that storage time and temperature interact to affect the amount of growth that would occur in foods that support growth. |
| Assumptions, Variability, Distributions | Non-U.S. pasteurized milk may not have same variability in contamination since pasteurization methods are different. FDA/FSIS used U.S.-only data to calculate the detection rate and average contamination level for pasteurized and non-pasteurized milk, but variability in the distributions came from U.S. and non-U.S. data. The assumption of similar variability may not be supportable. | Geographic weighting that reduces the impact of non-U.S. data was implemented in the 2003 risk assessment. |
| Assumptions, Weight, Distributions | Weights assigned to upper tails were up to 2.5 x 107 times larger than lower end. FDA/FSIS assigned weighting to data in proportion to reported concentration. This is incorrect because samples with high numbers or with bins with large endpoints are over-emphasized. Recalculate without weighting yields different distributions, e.g., 19 times lower 99 percentile in one example. It is more appropriate to state that weighting can yield much higher risk estimates than those derived from non-weighted data. | The weight referred to was a function of dose of 10 0.25. The revised procedure for incorporating quantitative information into the 2003 risk assessment has made high dose weighting unnecessary. |
| Categories | Current categories do not highlight food or processing characteristics. Regroup foods according to their characteristics and processing/handling. | This has been done, to the greatest extent feasible, based on available data. We have created food categories, which consider processing and food composition characteristics. Pertinent characteristics of the food that may have contributed to the contamination of a food category at retail are discussed in the technical document. |
| Categories | Focus on foods associated with Listeria monocytogenes, not food categories. Such groupings are inappropriate, introducing variability and uncertainty--lack of data is no excuse. | The goal of the risk assessment was to evaluate which foods that contribute to listeriosis cases. Individual foods were grouped into 23 food categories to accomplish this goal; in part, because insufficient data was available for individual foods and risk assessments for each of the over 640 ready-to-eat foods would be extremely complex. |
| Categories | It is recommended that the categories be split out, and to categorize foods separately for effective risk assessment and risk management. | The food categories used in this risk assessment were broad and the modeling techniques included consideration (as much as possible) of the variations within the food categories, including use of antimicrobials. |
| Categories | Low risk food items should not be included in the assessment. Remove frozen or acidified foods. | An objective of the risk assessment was to determine which foods are not contributing to listeriosis. This provides a quantitative estimation that they are not a problem that reinforces the qualitative judgment. |
| Categories | Regroup cheese according to ability to support Listeria monocytogenes growth. | Cheeses were regrouped according to ability to support Listeria monocytogenes in the 2003 risk assessment. |
| Categories | Divide Heat-Treated Natural Cheese and Process Cheese food category to "heat treated natural" and "pasteurized processed" cheese. Use of pasteurized milk data for distribution of processed cheese results in increased uncertainty. | The cheese categories were reorganized in the 2003 risk assessment, and the heat-treated natural cheeses are now grouped in either Soft Unripened Cheese or Soft Ripened Cheese food category. This risk assessment included some contamination data for processed cheese so surrogate data were not used. Processed cheese had very low risks because they do not support growth; further separation would not provide significant additional information. |
| Categories | Remove queso asadero and queso chihuahua from fresh soft cheese since they are firmer, drier cheeses. | Queso asadero and queso Chihuahua were removed from the Fresh Soft Cheese food category in the 2003 risk assessment and placed in the Hard Cheese food category as appropriate. |
| Categories | Rename category "fresh soft cheese" from "unpasteurized milk" to account for high contamination level. Also, only consider products made in "legally registered and approved establishments." | The specific recommendation was not feasible. However, a "what if" scenario analysis was conducted to evaluate the impact of higher contamination levels on the predicted risk attributed to fresh soft cheese. (See Chapter VI. 'What If' Scenarios.) |
| Categories | Deli meats contain products that differ substantially with respect to matrices, characteristics, production and handling. | Yes, variation in these categories includes food products that were different in regards to matrices, characteristics, production and handling. The assessment has captured that variability to the extent possible. |
| Categories | Many deli meats are either frozen, have a kill step after packaging, or inhibit Listeria monocytogenes growth. They should be separated since they are low-risk products---relates to placement of foods within categories (splitting Deli Meats food category before analysis). | Since data was at retail this was inherently captured in the data set and would thus be captured in the variability and uncertainty. Contamination at retail is an important factor that may over-ride processing factors. |
| Categories | Deli salads contain products that differ substantially with respect to matrices, characteristics, production and handling. | Yes, variation in these categories includes food products that were different in regards to matrices, characteristics, production and handling. The assessment has captured that variability to the extent possible. |
| Categories | Potato salad should be moved to Deli-type Salads food category. | Potato salad was moved to the Deli-type Salads food category in the 2003 risk assessment. |
| Categories | There is great variety in each of the seafood categories (i.e., different characteristics, handling, consumption, etc.,). Many examples are given, but it is unfair to assume similar patterns of contamination for all foods in category. Some ready-to-eat seafood is cooked, frozen, hand harvested, and etc., which impacts contamination; they should not be pooled together. | We have created food categories, which consider processing and food composition characteristics. Pertinent characteristics of the food that may have contributed to the contamination of a food category at retail are discussed in the technical document. |
| Categories | Hot and cold smoked seafood have differences in storage time, distribution practices, shelf life, and consumption patterns. | Available data including consumption patterns would not allow the differentiation. Furthermore, the data suggest pre-contamination after processing tends to limit the reduction in risk achieved by hot smoking. |
| Categories | Vegetables are ranked as low risk even though there is a high level of contamination of sprouts. Therefore, this should probably be subdivided. | In using a food category approach, there will be some foods that will not ideally fit that category perfectly. There were insufficient data on the extent of Listeria monocytogenes in sprouts to warrant its inclusion as a separate food category. |
| Categories | Divide vegetables into raw, pickled, and dry. Exclude soy products (eaten hot). | The Vegetables food category was revised to exclude pickled and dried vegetables, and soy products. |
| Categories, Assumptions | Consider if grouping foods in categories affects risk estimates (i.e., see if taking foods out of current groupings affects risk). | A risk parameter is always for a defined population, and this must be considered in interpreting the risk assessment. |
| Categories, Assumptions | Combining data across broad categories will not compensate for lack of deli salad data. | Newly available data on deli salad contamination and Listeria monocytogenes growth were incorporated into the 2003 risk assessment. The existence of growth and non-growth salads was recognized. (See Chapter III. Exposure Assessment, Modeling: Growth Between Retail and Consumption; and Chapter V. Risk Characterization, Food Category: Deli-Type Salads section.) |
| Categories, Contamination | Cabbage should not be in the Vegetables food category; listed only because linked to cabbage in slaw. Also, studies indicate Listeria monocytogenes grows well in refrigerated cabbage. | Cole slaw was moved to the Deli-type Salads food category. |
| Categories, Data | Re-categorize the ready-to-eat foods based on characteristics associated with contamination or growth of Listeria monocytogenes such as pH. Current categories are too broad, e.g., deli salads with and without meat and/or seafood/vegetables, vinegar vs. mayonnaise. | Consideration was given to the balance between categories, the availability of data, and the number of categories that can be dealt with. Every new category would need specific data for every step of the risk assessment from consumption to contamination to growth rates. For example, one could say that normal and low salt hams would have different growth rates and should be separated. Ultimately, it was decided that this risk assessment should be "broad" in its approach to facilitate interpretation. Focusing on how specific products are produced can be done in future risk assessments. |
| Categories, Data | Fruits category should include dry, fresh, frozen. | The Fruits food category includes raw and dried in the 2003 risk assessment. Consideration was given to the balance between categories, the availability of data, and the number of categories that can be dealt with. Every new category would need specific data for every step of the risk assessment from consumption to contamination to growth rates. |
| Categories, Data, Assumptions | National Food Processors Association (NFPA) has data on deli salads. The growth rate for deli meat should not be used for deli salads, which over-estimates risk in this matrix. | Newly available data on deli salad contamination and Listeria monocytogenes growth were incorporated into the 2003 risk assessment. (See Chapter III. Exposure Assessment, Modeling: Growth Between Retail and Consumption; and Chapter V. Risk Characterization, Food Category: Deli-Type Salads section.) |
| Categories, Growth | Divide Fruits category by pH, since low pH fruits do not support growth. Exponential growth rate from vegetables should not be used for fruits with low pH. | More data on fruits would be needed to further divide the Fruits category. The vegetable data was not used in the 2003 risk assessment. |
| Categories, Growth | Separate deli meats components, since not all deli meats support growth. | Dry fermented sausages were separated from other deli meats. The variability of the products in the Deli Meats food category is captured in the measures of variability and uncertainty. Further separation of deli meats would be better examined in subsequent risk assessments, specifically focused on the manufacture of these products. |
| Categories, Matrix | Group foods according to matrix, i.e., freezing, heating, or preparation. | The categories were based on matrix and growth characteristics of Listeria monocytogenes. Frankfurters had special consideration for consumer freezing and cooking. Generally, the level of detail was appropriate to make the desired inferences. |
| Categories, Proxy Data, Uncertainty | Some food categories are not uniform, especially vegetables. Grouping diverse foods obscures factors associated with Listeria monocytogenes risk reduction. Applying proxy data increases the uncertainty. This may be unavoidable because of data limits, but may not highlight unique characteristics relevant to risk management. Food categories contain foods that differ by characteristics, by processing, handling, diet consider. | Use of proxy data have been largely eliminated in the 2003 risk assessment (and completely eliminated for contamination). Each category would still have to be interpreted with the understanding of the specific foods that comprise the respective category. A more detailed examination of a specific food category would require a product-specific risk assessment, which was not the purpose of the current work. |
| Consumption | Most cases are not related to foodservice since they do not occur as outbreaks. Therefore, it is incorrect to state in this risk assessment that increased consumption of food from outside of the home or ready-to-eat foods is causing slowdown in Listeria monocytogenes reduction. | We agree that the consequences of the shift to consumption outside of the home are not known. In the 2003 risk assessment, risks in a food service were assumed to be comparable to those in home preparation. Surveys have shown that similar food handling problems are found in both places. A contaminated food in a restaurant would still be most likely to result in a single sporadic case rather than an outbreak. |
| Consumption, Categories, Distribution | Bi-modality may result from differences in consumption of aged cheese within the group, not because there were a high percentage of samples without Listeria monocytogenes. The concern with aged cheeses is that grouping with medium serving size of 27g obscures vastly disparate consumption patterns (e.g., many portions are Parmesan, but much larger amount is cheddar). Non-uniformity of category can affect distribution. | This is correct, and as a result a distribution of serving sizes was employed, rather than a single value for the entire group. Non-uniformity within a category for any factor will widen the distribution. With the improved data used in the 2003 risk assessment, the uncertainty in the risk assessment was greatly reduced. |
| Consumption, Cooking | Steamed or boiled cooked ready-to-eat crustaceans that are eaten hot were excluded. Much of this food is eaten hot, thus not risk. | This is a valid point. Unfortunately, the consumption databases did not ask whether shrimp were eaten hot. As a result, there may be some inaccuracies for cases per annum but the risk per serving would still reflect the risks for the unheated shrimp, which is of most concern. |
| Consumption, Cooking, Data | Use AMI data for frankfurter storage time and consumer behavior. (AMI data show that 7% of frankfurters are consumed without reheating.) Modify distribution to 1-6% uniform from 1-14% uniform. | The new American Meat Institute (AMI) survey was used as the basis for consumer handling of frankfurters (i.e., frankfurters eaten without reheating). The percentage of non-frozen frankfurters that were not reheated was represented by a triangle distribution of 4, 7, and 10. |
| Cooking | Model used to cook frankfurters is appropriate for risk management. Supports use of 1-6% versus 1-14%. | We concur. |
| Consumption, Data | Indicate serving sizes used to calculate data in Tables III-5 and III-11. | The serving sizes are distributions that are described on Table III-3. A graph is in Appendix 5. |
| Contamination | This risk assessment does not consider how food became contaminated, unlike other risk assessments. The focus on retail data misguided because the majority of outbreaks are associated with processing and management. Outbreaks associated with retail and restaurants most often occurred with already contaminated foods. | The design of the risk assessment was specifically developed to compare the risks associated with different classes of ready-to-eat foods and was extensively reviewed as providing the appropriate approach for addressing the stated purpose of the work. |
| Contamination | Data from outside the U.S. may have different contamination frequencies since processing varies. Do not assume that contamination distributions are the same. Examine extent to which variation in a food type (particularly if level of contamination in U.S. is lower) reflects true variation in part food or reflects different processing practices and country customs. | Weighting was employed to give greater impact to the current U.S. food supply in the 2003 risk assessment. Imported foods are a significant portion of the foods consumed within the U.S. However, countries such as Western Europe, Japan, Australia, and Canada are assumed to be similar to the U.S. |
| Contamination | Data more than 10 years old do not show the recent reduction in Listeria monocytogenes illness. Therefore, do not use older data. | A study date weighting system was implemented that gives greater importance to more recent studies. |
| Contamination | The importance of foreign contamination data should be proportionate to the consumption rate. | The U.S. food supply includes many imported foods. Geographical weighting was used to reduce the impact of contamination data from other countries. To ascertain the fraction of servings from individual countries is beyond the capabilities of the current risk assessment, both in terms of data availability and methodology. |
| Contamination | Foreign contamination data are a poor proxy for U.S. cheese; gives misleading estimate of risk of U.S. cheese. | For the 2003 risk assessment, studies were weighted in consideration of the geographic location. Less weight was given to countries that do not export foods to the U.S. |
| Contamination | Foreign manufacturers have different processing conditions that may result in higher contamination, skewing data. Do not use foreign data as proxy for contaminated levels in pasteurized milk in the U.S. Foreign manufacturers may have higher contamination levels. | Countries such as Western Europe, Japan, Australia, and Canada are assumed to be similar to the U.S. Using only U.S. data would be preferred, but in the 2001 draft there was insufficient data from the U.S. As a consequence of these data gaps, surveys were initiated to address this problem. The IDFA has provided new U.S. data that comprises the majority of samples. |
| Contamination | Distribution is based on contamination of fresh soft cheese made with unpasteurized milk. Fresh soft cheese must be made from pasteurized milk in the U.S. Omit this category until data are available. | The 2002 NFPA study provided recent contamination data for fresh soft cheese. The data reflected the fact that the majority of fresh soft cheeses are made from pasteurized milk. Some fresh soft cheeses are made from unpasteurized milk, and 'what if' scenario calculations were conducted to assess the impact of those cheeses. (See Chapter VI. 'What If' Scenarios.) |
| Contamination | Recent NCI study of soft-ripened cheese from pasteurized milk has a contamination rate of 0.06%. | The recent NFPA study had approximately 1% contamination. All recent studies are included in the 2003 risk assessment. |
| Contamination | The presence/absence data for Listeria monocytogenes in ice cream were provided by the Industry Council for Development of the Food and Allied Industries (ICD) for FAO/WHO Exposure Assessment of Listeria monocytogenes in ready-to-eat foods. Can you provide more recent contaminant level (enumeration) data? Also, new industry ice cream (and frozen dairy products) contamination data (from the International Ice Cream Association) are lower (0.18%) than data used in this risk assessment (0.7%). | Ice cream contamination data is now study date weighted in favor of data currency. The 2003 risk assessment includes more recent contamination data. (see Appendix 7.) |
| Contamination | Use more recent FSIS data for ready-to-eat meat and poultry. | The recent FSIS data were incorporated into the 2003 risk assessment. |
| Contamination | The risk of pasteurized milk is over-estimated since pasteurization kills pathogens. | Pasteurization may not kill all pathogens. However, more important is the frequency of post-pasteurization recontamination. In the 2003 risk assessment, the predicted per annum risk is not matched with an equivalent U.S. epidemiological record. Advanced epidemiologic and scientific investigations are needed to either confirm the predictions of the risk assessment or identify the factors not captured by the current models that would reduce the predicted relative risk. |
| Contamination | The contamination in unpasteurized milk is probably under-estimated; should not assume that competition from other micro-organisms will result in a decrease in Listeria monocytogenes over time. Rather than base contamination level of unpasteurized fluid milk at retail on assumptions about competition and limited data, instead base on data for pasteurized milk at retail, correlated with limited unpasteurized fluid milk data. | In the 2003 risk assessment, the contamination level for unpasteurized milk is 4.1% compared to 0.35% for pasteurized milk. In the 2003 risk assessment, the same exponential growth rates, maximum growth levels and storage times were used for both pasteurized and unpasteurized milk, based on published scientific investigations. |
| Contamination | Level of imported milk is 0.03%. Stating it is less than 1% is misleading | This percentage was deleted from text. |
| Contamination | Legume and vegetable sprout data should have been given more emphasis---treat sprouts separately. | Diversity within a food category is accounted for, however, the fact that certain foods may be at the extremes of the diversity needs to be considered when interpreting the risk assessment. Certain foods such as spouts may merit a specific product pathway risk assessment in the future, but this would require data on contamination at retail and frequency of consuming raw sprouts. |
| Contamination, Assumptions, Data, Categories | If climate in non-U.S. country is different then the implications should be considered and discussed. In some food categories, all data are non-U.S. or include data from countries with different climates. The relevance to the U.S. food supply should be considered. | The initial data set was contamination at retail; therefore, the U.S. would receive the effect of the local climate if a food were imported. Countries were weighted for each food category, depending on the importance of the food and the country of source. The growth was modeled using only U.S. refrigeration temperature data. |
| Contamination, Assumptions, Data | Excluding non-U.S. data for goat and feta (cheese) results in upper percentiles that are orders of magnitude lower than this risk assessment. | Goat and feta cheeses are no longer a separate category, and their contamination data are included with the Soft Ripened Cheese category. Cheeses from countries that do not contribute to the U.S. food supply are given low weightings, and the data set now includes the large recent U.S. survey (NFPA) of these cheeses. |
| Contamination, Data | The data in this risk assessment compared studies published pre- and post-1993. (Why is 1993 a dividing year for data?) The increase in the frequency of detection and problem awareness may be related to improvements in the detection methods and targeted sampling. As such, increased Listeria monocytogenes frequency post-1993 does not necessarily indicate a higher incidence of Listeria monocytogenes in the food supply. For some food categories (e.g., cooked ready-to-eat crustaceans), contamination levels are actually lower in post-1993 studies. Using pre-1993 data may over-estimate risk. Use post-1993 data for more accurate assessment. | A search of the published literature revealed that many of the studies were conducted in the late 1980s and early 1990s. From the published literature, it was difficult to ascertain the extent that improved sanitation and other control measures implemented by the food industry have reduced the frequency and level of contamination since 1993. Since some food categories had little data, which would result in a biased estimate, the overall trend in contamination for all of the food categories from before 1993 to after was obtained and applied to these data sets. (See Chapter III. Exposure Assessment, Food Contamination Data section.) The purpose of the pre- and post-1993 comparison was to assess any bias that may have been introduced unintentionally due to "study date." This has been dealt with in a different manner in the 2003 risk assessment through the inclusion of "study date" weighting system, which was employed to give recent data more influence on the contamination distribution. Post-1993 data is not available for all food categories. In addition, a correction factor was applied to anticipate reductions in prevalence estimates if new data were available for categories without new data. |
| Contamination, Data | Using only post-1993 data shows the 99th percentile is 5.6x lower for frankfurters than when all data are used. There is a concern in this risk assessment of similarity between pre- and post-1993, because some post-1993 publications contain pre-1993 data. Modeling change with only post-1993 data also changes the relative risks for frankfurters. | A weighting system was employed to give recent data more influence on the contamination distribution. Post-1993 data is not available for all food categories. In addition, a correction factor was applied to anticipate reductions in prevalence estimates if new data were available for categories without new data. Most of the frankfurter contamination data is from FSIS (2000 and 2001). Knowledge of when the samples were collected vs. the date of the publication would be the same for all food categories. |
| Contamination, Data | Use of pre-1993 data over-estimates predicted fresh soft cheese relative risk. | A weighting system was employed to give recent data more influence on the contamination distribution. With the inclusion of the newly available NFPA data, the majority of the contamination data set is comprised of recent data and has the most impact on determining the distribution. |
| Contamination, Data | Kozak (1996) pasteurized milk data is from the late 1980's and is outdated. The risk assessment should reflect when data was collected, not when published (regarding pre-1993/post-1993 split). | A weighting system was employed to give recent data more influence on the contamination distribution. For most studies, it is not known when data were actually collected. Even with the delay in publishing these data, Kozak data were not given full weight since these were not the most recent data. |
| Contamination, Data | Include the cheese data provided by National Cheese Institute (NCI) in the revised risk assessment. It includes both industry wide data on many cheeses and one manufacturer data on soft ripened cheese. | The data from NCI were included in the 2003 risk assessment. |
| Contamination, Data | Use new NFPA data on deli meats, deli salads, fresh soft cheese, soft mold-ripened and blue-veined cheese, vegetables, seafood salads, and smoked seafood. | The newly available 2002 NFPA retail study data were incorporated into the 2003 risk assessment. |
| Contamination, Data | New industry contamination pasteurized milk data is lower (0.018%) than data used in this risk assessment; data set includes one positive with enumeration. | The data set from the International Dairy Foods Association (IDFA) was used in the 2001 draft risk assessment and also in the 2003 risk assessment. |
| Contamination, Data | Older Dry/semi-dry fermented sausages data should be weighted; the recent data show a 5-log reduction for E. coli 0157:H7 for product produced in the U.S. | A weighting system was employed to give recent data more influence on the contamination distribution. The dry/semi-dry fermented sausages data included large surveys by FSIS in 2000 and 2001. Excluding certain data for one food category but not another would not be justified. |
| Contamination, Risk, Rank, Data | The risk per serving in deli meats is 400 times higher in risk assessment than when NFPA data were used. The relative rank changed sharply from 4 to 16 on a per serving basis, and from 1 to 13 on a per annum basis. | For the 2003 risk assessment, a different approach was used to estimate the distribution contamination curves; this approach yields a more continuous uncertainty distribution. The NFPA data were not available for the draft 2001 risk assessment. New data for many food categories were used in this risk assessment. |
| Contamination, Uncertainty, Foreign Data | Wide variation between studies in high Listeria monocytogenes occurrence levels, which contributes to the uncertainty. This may reflect different handling practices outside of the U.S. As such this may over-estimate the risk to U.S. consumers. | The uncertainty is acknowledged in the assessment and represented in the results through the uncertainty analysis component. For the 2003 risk assessment, the contamination data were weighted to try to more appropriately represent current U.S. conditions. However, Western Europe, Japan, Canada, and Australia are probably comparable to the U.S. It is difficult to use the available data to prove that the U.S. industry is more stringent. |
| Contamination Data | Quantitative data is hard to come by because of zero tolerance policy. | FDA/FSIS supports the need for systematic, regular collection of levels of L. monocytogenes in foods. |
| Contamination Data | Data in risk assessment should reflect experience of mainstream commercial food processors and purveyors, not small producers with problems. | The contamination data were collected from diverse sources, generally at retail. Most likely, the prevalence of retail samples reflects the respective prevalence of different classes of manufacturers. |
| Contamination Data | The contamination data come from diverse sources, may be out of date (with respect to food processing and handling practices), are largely nonquantitative, and do not specify the variables in handling (e.g., duration of time held at retail or distribution before sampling). | Each of the statements are correct, however, by considering a broad range of data with appropriate weighting, this risk assessment does provide a "national profile" of what exists at the retail level. |
| Cooking | Undercooking food can cause illness. One shouldn't assume that cooked foods have low likelihood of containing Listeria monocytogenes. | The cooking model employed in the 2003 risk assessment took into account the potential impact of different cooking times and temperatures. |
| Cooking | How were numbers for the triangular distribution assigned? Were they taken from Juneja (et al., 1997)? This is a different product; frequency distribution should not be applied to frankfurters. | Yes, the numbers assigned for the triangular distribution were taken from Juneja (et al., 1997), because inadequate data were found with which to directly model thermal inactivation in the frankfurters that were cooked. Although this is a different product, a hamburger study was used because it was the closest available analog for which data are available. (See Chapter III. Exposure Assessment, section Modeling: Thermal Inactivation.) |
| Cooking, Assumptions | One cannot assume that Listeria monocytogenes has similar thermal resistance to E. coli O157:H7. Why not use Listeria monocytogenes inactivation data? This needs justification. | We acknowledge that data on the D value of Listeria monocytogenes is available. However, the amount of thermal inactivation is not just the D value of Listeria monocytogenes vs. E. coli O157:H7. Heat penetration and the thermal profile within the product are also very important. We are not aware of data on thermal properties, heating rates, temperatures, and time that would be needed to make such a model. There are several different ways to cook frankfurters, each requiring the aforementioned data plus frequency of cooking method. This approach gave an estimate from a meat product and vegetative bacteria. |
| Cross Contamination | Collect data on handling practices to determine effect of cross contamination | Data on cross contamination were not adequate to put this factor into the model. There is considerable research activity in this area and it may be possible to consider cross contamination in the future. |
| Cross Contamination | Twenty percent of household patient-contacts are asymptomatic Listeria monocytogenes carriers; therefore refrigerator items that are positive for Listeria monocytogenes does not mean contamination resulted from processing or production failure. Table II-3 implies sporadic cases were caused by foods with Listeria monocytogenes found in them, but these data may reflect person to food transmission or cases may reflect person-to-person transmission. | This risk assessment is very strong on the point that proper post-production storage is an important component in preventing listeriosis. The outbreak data are used only to illustrate the widespread occurrence of Listeria monocytogenes; the data are not used in the risk assessment calculations. However, the epidemiological data strongly indicate that listeriosis is predominantly foodborne and not transmitted person-to-person (e.g., physical contact, sneezing, bodily fluids, etc.,). |
| Cross Contamination | One must consider cross-contamination, otherwise uncertainty is high; challenge assumption in exposure assessment that food categories contribute through direct consumption, while epidemiological data suggests cross-contamination plays major role. | It is recognized that the CDC study linking cooked chicken is likely a result of cross contamination. However, only recently has any data that quantitates cross-contamination become available. No data on frequency of cross-contamination or subsequent growth is available that would permit modeling. |
| Cross-contamination | Illnesses attributed to retail contamination may have resulted from cross-contamination. This possibility invalidates or argues against adjustment of data to retail levels. | The number of cases that result from cross-contamination is unknown. The use of retail data inherently takes into account contamination prior to and within the retail environment. For those food categories where data from production samples were used and adjusted to levels expected at retail, the data would not inherently include the impact of cross-contamination. |
| Data | Use data from the FAO/WHO Exposure Assessment and Hazard Characterization for Listeria monocytogenes in ready-to-eat foods. Compare assumptions, approaches and outcomes. | This risk assessment was conducted prior to the FAO/WHO project, even though the latter has become public first. The FAO/WHO assessment was developed for different purposes than the FDA/FSIS assessment; however, the international assessment is largely based on the U.S. evaluation. This includes a high degree of overlap in the exposure data employed. |
| Data, Assumptions | How did agencies treat data from sample sizes smaller than 25 g, particularly for quantitative enumeration studies? How does this affect contamination levels within and between food categories? Risks associated with those foods with the largest number of data points resulting from smaller sample sizes could be under-estimated. | The contamination distributions include samples with 103 to over 106 cfu/g. The difference between 1 cfu/25g and 1 cfu/10g is not a major source of uncertainty. |
| Data, Contamination | Contamination studies used here may be biased and not represent random sampling, e.g., Eklund study (et al., 1995) was from smoked seafood plants that were known to have Listeria monocytogenes problems. This skews contamination frequency data. | Many studies were available for smoked seafoods. Eklund (et al., 1995) is a small part of presence/absence data and represents only a fraction (less than 1%) of the data points comprising the entire data set. |
| Data, Rank | Substantial data uncertainties, data quality issues, and assumptions have significant impact on rankings. Changing data will alter rankings. | The new contamination data and other changes did reduce the uncertainties in this risk assessment compared to the 2001 draft. A risk assessment, just like a subjective judgment, depends on the quality of the data that is available and interpretations may change with additional information. However, this is also transparent by articulating the uncertainty of the measures. This risk assessment does provide additionally evaluations of the differences among the rankings. |
| Distribution | How does the choice of frequency distribution affect the final outcome? | The choice of the frequency distribution has a big impact on the final outcome. For example, the triphasic uncertainty distributions employed in the 2001 draft risk assessment resulted from the three different frequency distributions used to describe the Listeria monocytogenes concentrations. In the 2003 risk assessment, the degree of uncertainty was reduced by the use of lognormal distribution exclusively to describe Listeria monocytogenes concentration frequency; however, the range of parameter values employed still expresses considerable uncertainty. |
| Distribution, Cooking | Uniform instead of triangular distribution should be used for frankfurters consumption data. | A uniform distribution has an emphasis on the extremes. A triangular distribution was used for frankfurters eaten unheated (since there was evidence that there is a central tendency). |
| Distribution, Data | Why was a +/-20% uniform distribution used for the most frequent value and a +/-50% uniform distribution used for maximum value for post-retail storage data? | Like the frequency distributions themselves, the magnitude of the uncertainty and the central value are products of consensual judgment. The uncertainty at the maximum value is greater since these values are (by definition) very rare. |
| Distribution, Model, Uncertainty | This risk assessment did not provide goodness-of-fit for distributions; it is important to provide goodness of fit measure for individual distributions (not just ranking them or giving percentages of use) so that reader can judge uncertainty of individual fits. | Goodness-of-fit statistics are now reported in the Appendix 5. |
| Distribution, Model, Uncertainty | Parametric distributions used to describe sparse data sets introduce uncertainty; it is important to provide goodness-of-fit measure for individual distributions (not just ranking them or giving percentages of use) so that reader can judge uncertainty of individual fits. | The results in the 2003 risk assessment emphasize the medians along with the 5th and 95th percentiles more than the 2001 draft. As such, this should offer the reader a better perspective on the final uncertainty ranges. Goodness-of-fit statistics are reported in the Appendix 5. |
| Distribution, Storage | This risk assessment used cumulative instead of BetaPert to estimate concentration of Listeria monocytogenes in frankfurters after storage time. | The distributions currently used for frankfurters are based on USDA and AMI data (the mean comes from the latter, the bounds from the former). |
| Dose-Response | Table IV-2, there are more data on virulence. Docket copy has references; may require incorporating new data in model, not just revising text. | The mouse data's function was to provide the initial shape and spread for virulence. Studies from three independent laboratories were used to establish the mouse dose response. This distribution is five logs in width and additional data will not change that. The most critical step in the dose-response modeling was to adjust the position of the curve so the calculated contamination matched the CDC's estimates for illness and death. |
| Dose-Response | New data is being generated with transgenic mice that will reduce uncertainty. | Transgenic mouse model has great potential to increase the relevance of mouse oral dosing model to human illness, however this data is not yet available. To date, no testing of a large number of Listeria monocytogenes strains from food, outbreak or other sources in this model has been undertaken. Practical consideration for developing the model for large-scale studies would be availability and cost of transgenic mice. Alternatively, use of the guinea pig as a model (e.g., guinea pig shares critical e-cadherin residues for internalin A binding with humans) for oral infection may be more readily available. |
| Dose-Response | Stillbirth and neonatal infection in human cannot be predicted in mouse model. Use data from the University of Georgia on pregnant rhesus monkeys to adjust mouse data. | The University of Georgia primate study (Smith, et al., 2003) funded by FDA has not yet been completed, and will include only a relatively small number of monkeys. It is important to note that the mouse model provides only the shape of the dose-response curve and the measure of strain variability. |
| Dose-Response | Study human cases (epidemiology) of Listeria monocytogenes to get dose response data instead of extrapolating animal data; need to get data on humans before doing risk assessment. | There are only two outbreaks where food contamination, consumption, and attack rates are known. (see Appendix 9.) The incomplete data from these outbreaks does suggest that the numbers of Listeria monocytogenes consumed were large. |
| Dose-Response | The lack of data on Listeria monocytogenes serotypes results in over-estimation of potential illnesses. Assumption that all serotypes Listeria monocytogenes lead to listeriosis over-estimates potential rate illness and contradicts evidence that 3 out of 13 serotypes lead to 90% of food-borne listeriosis. | This risk assessment uses CDC's estimates of illnesses. If only a portion of the Listeria monocytogenes strains are causing the illnesses, then this risk assessment underestimates the virulence of those strains. Better knowledge on the virulence of individual strains is clearly needed. More information on the relative frequencies of contamination would also be needed to consider this. The estimates of virulence have uncertainties of two orders of magnitude to allow for strain differences. The rankings of the food categories would probably be unaffected by assuming only some strains cause the illnesses. (See Chapter IV. Hazard Characterization.) |
| Dose-Response, Assumptions | We agree with this risk assessment that there are not enough data to say whether specific strains cause disease or to change dose-response function. | The comment is appreciated. However, it must be noted that the 2003 risk assessment does explicitly recognize that there is a wide range in virulence among strains. |
| Dose-Response, Data | Update risk assessment to include new FoodNet data on illnesses. | For the 2003 risk assessment, four years (1998-2001) of FoodNet data were used. |
| Dose-Response, Transparency | How is the dose-response adjustment factor derived? | The dose-response scaling factor (new name for adjustment factor) is used to adapt the other portions of the model to the annual estimates of listeriosis derived from CDC FoodNet data. |
| Dose-Response, Transparency, Model | Information on the algorithms and assumptions used by program to fit dose response with mouse data was limited. Information is needed from FDA and FSIS--that is, more information needs to be provided for readers to use model. | The text of the 2003 risk assessment has been revised extensively and should be more transparent. The CD-ROM version of the risk assessment contains all of the files, which should therefore offer a greater understanding of the model. |
| Dose-Response, Uncertainty | The adjustment factor for mouse model is so high that it is a great source of uncertainty: mouse model and its relevance to listeriosis is one of greatest sources of uncertainty in this risk assessment. | The dose-response scaling factor (new name for the adjustment factor) is adjusted so that the amount of Listeria monocytogenes consumed leads to the number of cases determined by the epidemiological data. While it is acknowledged that there is much uncertainty, the mouse data comprises but a minor part. Additionally, the mouse model is not the only source of uncertainty contributing to the magnitude of the scaling factor. |
| Future | Follow up on high-risk categories and generate product/pathway-specific risk assessments for more effective risk management. | A product/pathway-specific risk assessment was not an objective of this risk assessment. However, continued attention to high-risk categories as well as the development of product/pathway-specific risk assessments is being considered. |
| Future | Conduct "process risk assessments" to determine effect of interventions. | The risk assessment design was appropriate for the task given the risk assessors. (See Chapter I. Introduction.) If asked in the future to examine risk reduction strategies for specific foods, then a product pathway analysis would be appropriate. |
| Future | The model should be able to perform sensitivity analysis to develop effective risk management strategies. | The complexity and method of calculating the risk assessment do not provide for simple tornado graphs and make traditional sensitivity analyses more difficult. The uncertainty distributions are described. The "what if" scenarios now provide one type of sensitivity analysis. This risk assessment does provide better information needed for broad risk management strategies among food categories whereas risk management choices within individual foods may require additional product pathway analyses. (See Chapter VI. 'What If' Scenarios.) |
| Future | Periodically update risk assessment with new data. | A risk assessment uses data available at the time to answer specific questions from the risk management team. New and/or additional questions may be posed by FDA or FSIS in the future that would lead to an updating of the risk assessment. New data naturally would be included in such updates. |
| Growth | Use the American National Standards Institute/National Sanitation Foundation (ANSI/NSF) Standard 75- 2000: Non-potentially Hazardous Foods test to determine if a product can support growth of Listeria monocytogenes to dangerous levels (e.g., limit the acceptable level of growth to less than two logs within the product's shelf, or to levels no greater than 100 cfu/g at time of consumption). | A protocol to implement a growth/no growth policy would have to specify the amount of allowable growth and the methods to determine that growth. Whether or not to differentiate between growth and non-growth foods or to allow a specified amount of growth is a risk management policy question and, as such, is not within the scope of this risk assessment. |
| Growth | How did FDA/FSIS adjust for differences in inoculum levels (from inoculum studies) within and between food categories in order to accurately model post-retail growth? | The assumption is that at the exponential growth rates are independent of the initial inoculum levels. This is generally assumed for modeling and the interpretation of any inoculated pack study. (See Chapter III. Exposure Assessment.) |
| Growth | We agree with FDA/FSIS that modeling refrigeration and storage time distributions independently would be inappropriate. High temperature and long storage time would cause products to spoil and would competitively inhibit Listeria monocytogenes growth. | An inverse correlation is included in the modeling to avoid extreme combinations of high temperature and long storage times. |
| Growth | Justify use of square root model to emulate decline, since the model has only been tested for growth. | Many inoculated pack studies in several of the food categories found slow rates of decline in the numbers of Listeria monocytogenes. To improve the accuracy of the modeling beyond that of considering "no growth," a simple model for decline was needed that would evaluate the effect of refrigeration temperature and smoothly integrate the samples that had growth with those that had declines. Because the square root mode was used for growth, and a negative parameter value decreased the populations and the model had temperature in the model, it was a logical choice for making an estimate. Previous research has found that rates of decline are faster as the temperature increases (Pathogen Modeling Program), which is what this approach does. (See Chapter III. Exposure Assessment, Modeling: Growth Between Retail and Consumption section.) |
| Growth | Modify Table A5.1.9 to limit maximum growth to 4 log cfu/g at <5°C. | There is literature data cited in Appendix 8 where growth exceeded 4 log at 5°C. For example, Pelroy (et al. 1994a) found growth to five logs at 5°C in smoked salmon. |
| Growth | The growth factor for cooked ready-to-eat crustaceans was inappropriate, and should be lower or none. Cooked ready-to-eat crustaceans are frequently stored on ice, which is also a critical point under HACCP inspection. | This was based on three studies-- all indicating relative rapid growth rates for this food category. The growth that was modeled was for home refrigerator storage, not retail storage, therefore, the impact of storage in ice would not be included in this risk assessment. |
| Growth | Growth rate in fruit is based only upon orange juice serum study. Higher pH foods, different sugar content, and etc., would yield very different growth rates. | Additional data were found, specifically on apple slices. A broad range for variance was used to encompass the diverse characteristics of fruits. |
| Growth | The use of deli meats growth for deli salads was not scientifically sound. (Deli meats and deli salads have different pH levels, water activity, and preservatives profiles.) No justification is given beyond absence of deli salad data. | Newly available data on deli salads were incorporated into the 2003 risk assessment, and surrogate data were not used. (See Chapter III. Exposure Assessment, Modeling: Growth Between Retail and Consumption; and Chapter V. Risk Characterization, Food Category: Deli-Type Salads section.) The previous model used deli meats because they are frequently ingredients in deli salads and would provide a microenvironment favorable for growth. |
| Growth | Only used some data from Dillon and Patel (1992); Docket copy references other studies with lower smoked seafood growth rates. Also, some data shows naturally contaminated smoked seafood grows more slowly than where smoked seafood is inoculated. | The data in Dillon and Patel (1992) had only single replicates, and was a very limited data set. We chose the portion of this data set that was considered the most relevant. |
| Growth, Assumption | Data on lag phase and cell viability are essential to valid calculations. Consider these factors in determining growth under various processing, handling, and storage conditions. | The risk assessment does not model the manufacturing process. The rationale for disregarding the lag phase is discussed in depth in Chapter III. Exposure Assessment. |
| Growth, Assumption | Using only U.S. data for pasteurized milk allows results in dramatic shift of predicted rank. Use of estimated exponential growth rate to determine levels at retail is wrong, leading to per serving risk 4000 times higher than without growth adjustment. Omitting growth rate adjustment changes risk from 10 to 18 per serving and from 3 to 17 per annum. | In the 2003 risk assessment, contamination data sets were weighted for survey size, study date, and country. There is also an extensive new contamination data set for milk. A new approach to modeling the distribution was used that reduced the uncertainties for the extremely high contamination was also employed. Omitting growth rate adjustment changes risk from 10 to 18 per serving and from 3 to 17 per annum is based on an erroneous calculation. The adjusted concentration in milk after 0.25 logs of growth is only 0.07 cfu/g, not 0.7 cfu/g. |
| Growth, Assumptions | The importance of assumptions about growth and the need to estimate Listeria monocytogenes levels accurately should be noted in this risk assessment. For example, the way the data was used in this risk assessment may have artificially inflated estimates of concentration levels at retail, for samples collected pre-retail, resulting in artificially inflated risk estimates for certain food categories. | The majority of data used in this risk assessment were from retail samples. When pre-retail data were used, expert opinions were sought on the likely conditions that these products would encounter. The contamination table (Table III-4) indicates what samples were taken pre-retail. Ignoring the potential conditions between manufacture and retail would have inappropriately deflated the values for the limited number of food categories where pre-retail data were considered an important source of information. |
| Growth, Categories | This risk assessment fails to consider different growth rates of Listeria monocytogenes in foods combined in specific categories. That is, for many categories, disparate foods are combined inappropriately (e.g., roast beef with poultry meats, sprouts and cabbage with vegetables, high pH and low pH fruits, and etc.,). | The food categories do consider product characteristics, for example deli meats vs. dry fermented sausages. There is a limit to the number of categories that can be created considering the complexity of the risk assessment and the need for data for each factor for each food category. Some distributions for growth rates are relatively wide but are determined by the diversity of the growth rates within a category. |
| Growth, Contamination | The Institute of Food Technologist (IFT) report (2000) indicates that cold smoking decreases Listeria monocytogenes, contrary to this risk assessment. | This risk assessment is not concerned with changes during processing. Retail surveys show the contamination at retail, and many studies show Listeria monocytogenes growth during storage of finished product. |
| Growth, Data | Reference articles observing inflated growth in inoculated pack studies compared to natural contamination of seafood. Advise not to use inoculated data as basis for post-retail growth estimates. | Growth rates are generally independent of contamination levels. There is very little natural contamination data to use. The scientific data employed is provided in Appendix 8. |
| Growth, Transparency | For time and temperature data, how were values interpolated from empirical distributions of the table of percentages in Table III-8? | The home refrigerator data (of Audits International) were used as a histogram, the frequencies in the table were assigned to the average temperature of that group. For example, 3% of the refrigerators were at 49° F. |
| Management | Knowledge gaps must be filled in before a response plan can be developed. | Unavoidably, knowledge will always have gaps. A risk assessment is intended to get the maximum value from existing data. The uncertainty allows the agencies to determine whether the data is sufficient to support their decisions. HHS and USDA have proposed short and long term initiatives to reduce listeriosis, which will be modified as new data becomes available. |
| Management | Labeling should be a new Listeria monocytogenes strategy to alert high-risk consumers of potential risk. | This approach is addressed in the HHS/USDA report, "Reducing the Risk of Listeria monocytogenes: Joint Report to the President." This report is available at: http://www.foodsafety.gov/~dms/lmriplan.html. |
| Management | The degree of variability and uncertainty should be considered before proposing new regulations based on risk assessment results. | FDA/FSIS agrees that variability and uncertainty should be considered in interpreting and using risk assessments. |
| Management | Eliminate "zero tolerance" for foods that do not present a risk of listeriosis. | The HHS/USDA report, "Reducing the Risk of Listeria monocytogenes: Joint Response to the President," explains the proposed action plans to reduce listeriosis. This report is available at: http://www.foodsafety.gov/~dms/lmriplan.html. |
| Management | Cite High Pressure Processing as an intervention method to reduce Listeria monocytogenes in food. | This risk assessment begins with foods at retail, and an evaluation of the impact of specific intervention methods is outside its scope. Additional risk assessments to evaluate specific interventions such as High Pressure Processing would require product specific pathway analyses. |
| Management | Omit feta cheese from FDA consumer food safety message. | The consumer messages will be re-evaluated in consideration of the re-organization of cheeses based on moisture content. |
| Management | Direct efforts to products that support Listeria monocytogenes growth. Ice cream and frozen dairy products do not. | The HHS/USDA report, "Reducing the Risk of Listeria monocytogenes: Joint Response to the President," explains the proposed action plans to reduce listeriosis. This report is available at: http://www.foodsafety.gov/~dms/lmriplan.html. |
| Management | Relative rankings have fundamental uncertainties that impede risk management, and they will change with new data and assumptions. | The 2003 risk assessment gives the measurement values and their uncertainties for risks per serving and cases per annum. The rankings are a tool to help communicate these results and it is recognized any ranking procedure looses information. The agencies (FDA and FSIS) have both types of information for their evaluation and use. |
| Management, Risk, Rank | Relative risk ranking does not give details to develop effective control strategies. More data are needed. | Evaluating specific control strategies was not an objective of this risk assessment. |
| Model | Overall, commend the risk assessment. | The comment is appreciated. |
| Model, Transparency | Where there is lack of data, this risk assessment is reasonable, transparent and conservative. It used distributions for key variables, rather than point estimates. It also identified explicitly and quantitatively data variability and uncertainty and areas where critical research was needed. Overall, this risk assessment is transparent and amenable to review and evaluation. | The comment is appreciated. |
| Modeling | Storage times and temperatures were not estimated for production or retail. | The model does not attempt to model the production process. However, because some samples collected during production were used to estimate Listeria monocytogenes concentration at retail -- an adjustment was made to the concentration associated with the prevalence value that was based on estimated growth. The storage times and temperatures used for this adjustment are listed on Tables III-6 and III-7. Foods were assumed to be sampled from retail cases without consideration to their retail storage times or shelf life. The data reflect a random sampling of what is purchased and there is no need to consider growth during retail storage. |
| Modeling | Compare current presence/absence approach with a different approach, i.e., estimate prevalence based on the number of positive/negative samples and the concentration based on quantitative levels in positive samples. This would alleviate the need for extra weighting step for data at higher concentration levels. Refer to FAO/WHO Exposure Assessment of Listeria monocytogenes in RTE foods. | Different approaches for evaluating the data were considered in this risk assessment. The present approach takes the size of the sample into account in evaluating the implication of prevalence assays on Listeria monocytogenes concentration values. The FAO/WHO Exposure Assessment used much of the same data as this risk assessment. (See Chapter III. Exposure Assessment, Food Contamination Data section and Modeling: L. monocytogenes Levels in at Retail section.) |
| Modeling | For presence/absence data, how was <0.04 cfu/g treated in the distribution? Which value or distribution was used? (For qualitative studies, if "absence" = 0.04 cfu/g, what value is given to "presence?") | When fitting the distributions, the data are converted to cumulative values (i.e., the fraction of values above or below a particular value is calculated). "Presence is  0.04 cfu/g, and "absence" is <0.04. The 0.04 cfu/g value (for 25 g samples) is used to place a prevalence value on a cumulative distribution; it is not a concentration estimate. |
| Modeling | Adjusting data for foods sampled at pre-retail does not consider factors that would impact the level at retail. Also, using post-retail data assumes Listeria monocytogenes was present on food at retail, which may not be the case or may be cross-contamination. Reconsider use of adjusted data for retail. | Some data sets were of contamination levels at manufacture. To include them with the majority of the data from retail samples, an adjustment for growth between manufacture and retail was necessary. Representative times and temperatures were chosen based on expert opinion, and a single point adjustment value was determined for each food category. (See Table III-12, and supporting text in Chapter III. Exposure Assessment, Modeling: L. monocytogenes Levels in Food at Retail section.) |
| Modeling | Do risk assessment for pooled data and compare to non-pooling. Also break some foods out of categories and compare to check grouping effect on risk estimates. | The approach to deriving Listeria monocytogenes concentrations has been revised. It would be possible to pool the results of the various studies instead of employing them separately to characterize an uncertainty distribution. Whether or not this is appropriate depends on the willingness to claim that each study reports a sample that is: a) perhaps analogous to the U.S. food supply, or b) partly analogous to the U.S. food supply. This need not be an all or none choice -- some further pooling could be considered without necessarily pooling all the data. This is potentially an analytically intensive project. The initial evaluations of this suggestion indicated that the gains achieved would not justify the degree of analysis required, and would substantially delay the publication of the risk assessment. |
| Modeling, Contamination | Bin size may give greater influence to points at upper end of distribution. | Since the dose range is much greater than the bin interval, the bin size should not have a greater influence on the upper end of the distribution. |
| Outbreak, Data | CDC data suggests outbreaks are common. Identifying each Listeria monocytogenes positive as a separate occurrence implicitly over-estimates the number of events that led to positives (i.e., number of episodes of contamination) and thus overstates risks per annum for some foods. | A small percentage of the total number of cases is associated with outbreaks. Since the assessment targets an annual case rate that represents a four-year average, it is only necessary to assume that the distribution represents average (i.e., a 4-year average) contamination rates. (See Chapter IV. Hazard Characterization, Dose-Response Adjustment Factor section; and the introduction of Chapter V. Risk Characterization.) |
| Outbreak, Data, Assumption, Contamination | The assumption that contamination distribution is relatively constant is not supportable, considering the outbreak data (e.g., pasteurized milk). | The assessment targets an annual case rate that represents a four-year average; therefore it is assumed that the distribution represents average (i.e., a 4 year average) contamination rates. (See Chapter III. Exposure Assessment, Food Contamination Data section.) Also, the predicted per annum risk is not matched with an equivalent U.S. epidemiological record in the 2003 risk assessment. Advanced epidemiologic and scientific investigations are needed to either confirm the predictions of the risk assessment or identify the factors not captured by the current models that would reduce the predicted relative risk. |
| Outbreak, Data, Model, Ranking | Use outbreak data to identify sources of pathogen in the food supply, to validate models and rankings, and to identify attack rate. | Outbreak investigations are not sufficiently complete to identify all of the source foods, particularly foods more likely to cause sporadic cases. |
| Risk | Non-reheated frankfurters are not included in the four interpretation/conclusion groups. They, as well as heated frankfurters should be in the "those that warrant identification of new approaches" because of potential, regardless of cooking, and through cross-contamination of other foods in kitchen. | Non-reheated frankfurters are now a separate food category and are given a complete discussion in the text of the risk assessment. (See Chapter III. Exposure Assessment, Modeling: Thermal Inactivation section.) Cross contamination could not be evaluated in this risk assessment because of a lack of information. However, the potential is recognized and is discussed more fully in this risk assessment. Even frankfurters that are reheated could be a source of cross contamination to another food prior to heating. |
| Risk | The risk assessment estimate of smoked seafood in Figure V-1 is inconsistent with CDC findings. | The risk assessment was specifically designed to start with contamination data and product characteristics, and predict risk of listeriosis. The results were then compared against the epidemiological record. Foods such as smoked fish, which are manufactured in relatively small lots and infrequently consumed, would not cause outbreaks that would be detected. These types of products would lead to sporadic cases, which are rarely traceable in epidemiological studies. The limitations in trace back are one of the reasons this risk assessment was conducted. |
| Risk, Data | The risk assessment may over-estimate the risk associated with seafood (e.g., smoked seafood may cause 16 cases per 100 million servings, and 32 annual illnesses), yet no culture confirmed cases in CDC database. | There have been no laboratory confirmed outbreaks involving smoked seafood in the U.S., however, there have been episodes reported internationally. (See Chapter II. Hazard Identification, Outbreak-Associated Listeriosis section, and Table II-5.) Foods such as smoked fish, which are manufactured in relatively small lots and infrequently consumed, would not cause outbreaks that would be detected. These types of products cause sporadic cases, which are rarely traceable in epidemiological studies. The limitations in trace-back are but one of the reasons this risk assessment was conducted. |
| Sensitivity Analysis | Hypothesis testing of grouping foods will elucidate uncertainties and data gaps. | Since the number of potential food groupings is innumerable, consideration of all of them would have made the risk assessment overly complex. The foods were therefore grouped into 23 manageable food categories. However, 'what if' scenarios were tested in the 2003 risk assessment that provided further insight into the relationships between contamination, growth rate, storage temperature, and storage time. (See Chapter VI. 'What If' Scenarios.) |
| Sensitivity Analysis | Test the influence of food matrix, packaging, and processing conditions to determine which foods do not support Listeria monocytogenes growth. | The risk assessment was not intended to model food production. It does indicate, however, the difference between foods that support or do not support growth. |
| Storage | Either adjust with expert judgment or don't use FSIS and Georgetown data. Preliminary data from survey of callers to FSIS Meat and Poultry Hot Line is unrepresentative because Georgetown survey provided only preliminary data, and the information from the hot line does not reflect the practices of the average consumer. The survey data should be adjusted based on expert judgments and average/mean expiration dates on prepackaged deli meats. | The newly available AMI survey data have been incorporated into this risk assessment. These data are not significantly different from the data provided by FSIS. |
| Storage | Use the new AMI data to generate new distributions of storage times to model these data. | The new AMI survey data are incorporated into this risk assessment. However, the AMI survey recorded 'average' storage times across households. It therefore dose not represent the distribution of storage times for individual servings. |
| Storage | The 10-20 day maximum storage time is too long for cooked ready-to-eat crustaceans, especially for cooked lobster and shrimp. | With the shape of the distribution used, only a very few samples would reach these times. Expert opinion indicated that a small percentage of consumers would store these foods for an extended period. |
| Storage | Consider the likelihood and duration of refrigeration and frozen storage of frankfurters. | New data on the refrigerated storage of frankfurters were included in the risk assessment. Consideration of the percentage of frozen frankfurters was also considered. (See Chapter III. Exposure Assessment, Modeling: Thermal Inactivation section.) |
| Storage | There is new data on consumer deli meats storage times. | The new AMI data were incorporated into the 2003 risk assessment. |
| Storage | The 180-day frankfurter storage time is believed to be an outlier. | The AMI data was used as the basis for a revised storage time distribution. This study asked consumers about their "average" storage times, it did not determine the times for individual frankfurters. Outliers do occur at a predictable frequency. This was the extreme example but there was no justification for dismissing the validity of the single data point. However, its impact on the overall distribution is minimal. |
| Storage | Frankfurter and deli meat storage times are probably under-estimated. The "moderate" time frame is inconsistent with use-by dates, which many customers exceed. The timeframe should be "long." | The moderate vs. long designations on Table V-5a are intended as qualitative aids to understanding the many factors in the risk assessment, and are arbitrary and based on expert opinion. (See Table III-5 for actual values used.) The designations had no influence on the calculations. Hopefully, the respective tables clearly indicate the actual values for any food category one would be interested in. (The data sets employed are presented in Appendix 8.) |
| Storage | Before risk assessment can be valid, accurate data describing holding times and temperatures is needed. Use real data on storage times for food in home; do not include estimates. | This risk assessment strived to use the best information or expert opinions available. Considerable effort was expended to get additional information on consumer practices for different food categories prior to developing this risk assessment. An uncertainty value was incorporated into the storage time distributions. (See Chapter III. Exposure Assessment, Growth Data section.) |
| Storage | New data: queso blanco normally eaten 2-3 days after buying. Queso blanco storage distribution should be minimum: 0.5, mode: 1-5, maximum: 30 days. | In the 2003 risk assessment, the times were adjusted to fit this new data. |
| Storage, Assumptions, Sensitivity Analysis, Rank | Change fresh soft cheese values to mode: 1-5, maximum: less than 30 days. Using indicated values in the storage distribution lowers the estimated per serving risk for the elderly population by a factor of 9. This result shows the impact of a small change in assumptions used by FDA and FSIS, and illustrates the need for an assessment of impact of the uncertainty in each input parameter (on the uncertainty of the derived risk estimates). | Showing that a change in an input will affect the output is no sufficient grounds for either changing or doubting the model. It is still necessary to argue that the input values should be changed -- i.e. the estimates should be shifted or the uncertainty bounds made wider or narrower. This is why an uncertainty analysis is more important than a sensitivity analysis. |
| Storage, Consumption | Much ready-to-eat seafood is frozen before consumption, which should be taken into account. Some storage time after retail may be frozen (e.g., finfish for sushi, cooked ready-to-eat shrimp), and should be reflected in post retail growth assumptions. | This was factored into frankfurters, but in seafood this could not be carried further for lack of data on amounts stored frozen for each food. |
| Storage, Data | The per-serving risk in frankfurters in this risk assessment is 27 times higher than when AMI data used. The relative rank changed from 8 to 15 on a per serving basis, and from 4 to 11 on a per annum basis. | The AMI data was used as the basis for a revised storage time distribution. However, this study asked consumers about their "average" storage times, it did not determine the times for individual frankfurters. |
| Storage, Data | The most likely storage duration time duration for frankfurters was modeled to be between 5 and 7 days, yet 88% modeled storage durations were longer than 7 days. The mean and median storage durations were 35 and 28 days, respectively. | This illustrates some of the characteristics of skewed distributions. The mean is much affected by a few high values; this is why the median is usually reported in the risk assessment to describe a distribution. The shape of the distribution is highly uncertain, particularly with the frequencies of longer storage times. The AMI data improved the distribution for frankfurters but there is still considerable uncertainty associated with our knowledge about consumer handling of all ready-to-eat foods. |
| Storage, Distribution, Temperature | For storage temperature, are the minimum and maximum temperatures the absolute values? | The storage temperature distributions are empirical -- the maximum and minimum values are taken directly from the Audits International data set. |
| Storage, Temperature, Model | Negative correlation between storage time and temperature was intuitively correct, but mathematically arbitrary. | There was uncertainty about the nature of the correlation; therefore a simple model with a large uncertainty range was employed. |
| Temperature | Why is T0 a point estimate and not a distribution? Or is it a distribution? If it is a point estimate, it is inconsistent with other choices. | There could have been a small uncertainty distribution added for T0 but the different sigmoidal models for the growth rates were a significant source of uncertainty in the estimate of the exponential growth rate. |
| Transparency | This risk assessment is reasonably transparent to the technical professional. | The comment is appreciated. |
| Transparency | Establish a mechanism for comments through JIFSAN Risk Analysis Clearinghouse. | Comments should be submitted to the public dockets. |
| Transparency | There are several inconsistencies in data described in the draft risk assessment. Examples include: inconsistencies in the summary concentration data vs. the published contamination data and the cumulative distributions used; Cortesi et al. 1997, gives same frequency at two different concentrations; and text has different numbers for Weibull-Gamma and Beta distributions than the table. | A detailed, critical review of this risk assessment was conducted to eliminate data inconsistency as much as possible. |
| Transparency | Can model determine which inputs affect the risk estimate the most, and what effect a change would have on predicted illness? | The scenarios that were added to this risk assessment should provide much the requested information. The structure and complexity of this risk assessment did not lend itself to simple sensitivity analyses and tornado plots. |
| Transparency | There seems to be more certainty in numbers at the high and low ends of the food categories than for the middle rankings. Instead of a numeric rating system, group according to High Risk, Low Risk, and Uncertain. | The 2003 risk assessment focuses more on the actual values and distributions. Hopefully, the uncertainties of the rankings are adequately demonstrated in the latitude graphs. In addition, examples of cluster analyses are provided to provide a potential qualitative grouping of food categories. Rankings, cluster analysis, and use of high/medium/low categories are communication tools. |
| Transparency | Do not revise the risk rankings; instead focus on risk per-serving and per-annum. As new data comes in and risk assessment is revised over time, revise risks rather than ranks. | Ranking is a communication tool, and, inherently some information is lost when one ranks. In addition to the rankings, the 2003 risk assessment offers the actual values (and uncertainties) for both risks per-serving and cases per-annum more prominently than the 2001 draft. |
| Transparency | How do you run the programs in the various spreadsheets? How can the outputs from the spreadsheets be linked? How can assumptions be modified? Which default settings can be changed in the spreadsheets? When must changes be made in the software code? | The descriptions in the 2003 risk assessment hopefully are more explicit about how the spreadsheets relate to each other. The modeling software on the JIFSAN clearinghouse website (http://www.foodriskclearinghouse.umd.edu) should be helpful to many people who wish to test different scenarios. Although portions of the previous model were written in Excel worksheet language, the 2003 risk assessment is almost entirely written in Excel Visual Basic for Applications. The worksheets are only used to store parameters inputs and to record the model output. As a result, the model's "user-friendly" software is much easier to follow, but modification requires knowledge of Visual Basic and the Visual Basic Editor. |
| Transparency | Provide additional explanatory text and instructions for use of this risk assessment (i.e., update and simplify Appendix 6, Software), and create modules that allow the user to look at data for specific foods. Also, create a mechanism for users to offer input on the model by submitting comments and/or data. | The 2003 risk assessment is almost entirely written in Excel Visual Basic for Applications. The worksheets are only used to store parameters inputs and to record the model output. As a result, the model's "user-friendly" software is much easier to follow, but modification requires knowledge of Visual Basic and the Visual Basic Editor. An abbreviated version of the model was placed on the JIFSAN Risk Analysis Clearinghouse website to allow interested parties to test changes of interest to them. |
| Transparency | Add more sub-results so others can recalculate. Also, clarify quantitative assumptions. | The CD-ROM (new version) contains all data tables. New, "friendlier" software should make process more transparent. |
| Transparency | Use of unpublished data is unacceptable. | All of the data (published and unpublished) sources are made available in the public dockets and are available for review. Although laboratory data from government laboratories oftentimes are not published, such data were considered appropriate and valid. |
| Transparency | It is unclear how to run "what if" scenarios. | The 2003 risk assessment includes some 'what-if' scenarios that will help illustrate the interactions of contamination, temperature, time, and growth rate on the rates of illness. A software model that allows scenarios for individual foods has been developed and is available on the JIFSAN clearinghouse website: http://www.foodriskclearinghouse.umd.edu. |
| Transparency | Appendix 5 is in black and white---can't identify which lines correspond to which model. | Some are not clear although the qualitative point being made by the graph is still evident. The electronic version of the risk assessment is in color, and available at www.cfsan.dfa.gov, www.fsis.usda.gov, www.foodsafety.gov, and www.foodriskclearinghouse.umd.edu. |
| Transparency | Charts with ranges of predicted risk per-serving and per annum should be moved to follow Tables V-2 and V-3 since the rankings are not hard numbers. | In the 2003 risk assessment, the figures with the predicted risk rankings per serving and per annum follow the corresponding tables containing the median, 5th, and 95th percentiles. (See Tables V-1 and V-3, and Figures V-1 and V-3). The tables for the predicted relative risk ranking per serving and per annum are Tables V-2 and V-4, respectively. |
| Transparency | Table A5.1.8 shows N=25, but in Figure A5.1.3 there are 28 data points. | The mean and standard deviation in Table A5.1.8 describes the data in a simple manner; it is not exactly what was used in the modeling. However, the N value and number of points should correspond. Additional data has been added for some food categories and this modeling method has been replaced. |
| Transparency | Why are there 15 references in Table A5.1.3 but 16 data points in Figure A5.1.2? | Each study can have several points on the graph if that study has more than one quantitative value. |
| Transparency | Figure A5.1.3, p. 47 (also see App. 5, p. 234) shows growth at 5°C. How are other temperatures included in the calculation? | Individual studies used different storage temperatures. To create the model, the growth rates were calculated for 5°C for all growth curves, which is on Figure A5.1.3. When the modeling requests another storage temperature, the same calculation is used to determine the rate of that temperature. |
| Transparency | For the data point at cumulative frequency of 0.93 in Figure A 5.1.2, where is the other 7%? | That point on the figure means in one study, 93% of the samples were negative at the specified detection level and 7% were positive. |
| Transparency | Table III-7 and A5.1.8 present the same smoked seafood data, however page 45 states that this mean and standard deviation weren't used, cumulative table of actual data points used instead. Delete Table A5.1.8. | The means and standard deviations were provided for comparison even though the modeling may differ slightly. |
| Transparency | For smoked seafood, two different sample numbers are given (71 & 309), and two different relative frequencies cited for Listeria monocytogenes concentration level of 0.04 cfu/g for Teufel and Bendzulla, 1993 study. | Some studies have more than one data set. Each would have a different fraction of samples positive at the same detection level (0.04 cfu/g) |
| Transparency, Data | Clearer explanation of how FDA/FSIS used the data is needed. | The modeling sections have been rewritten and, hopefully, are clearer. Examples have also been added to explain the distribution fitting for contamination. (Refer to Chapter III. Exposure Assessment.) |
| Transparency, Data | The underlying data should be available for review and evaluation. | The 2003 risk assessment includes the contamination tables. (see Appendix 7.) All data are available on CD-ROM with the model. |
| Transparency, Distribution | In general, provide more explanation for why certain distributions were chosen (e.g., uniform distribution for storage temperatures). Why use a uniform distribution instead of normal distribution to describe storage temperature? | In the 2003 risk assessment, more extensive explanations are given on why a particular distribution was selected. A histogram of the actual data was used for storage temperature. The data were roughly normally distributed with a mean of 39°F. Generally, uniform distributions were used to describe the degree of uncertainty about a parameter value (most like storage time) that described variation. The adjustment for growth pre-retail was a uniform distribution with a narrow range whose purpose was to estimate a point adjustment value. |
| Transparency, Model, Distribution | It is not clear how ParamFit derives the parameters of some of the distributions. | The documentation for ParamFit is included in Appendix 6. It is similar to other algorithms that fit equations to data sets that used a series of approximations that get closer to the best values for the parameters with each iteration. |
| Transparency, Temperature, Storage | What are the parameters of uniform distribution (Table III-6)? What were the minimum and maximum storage times? | A uniform distribution is defined by its low and high values, which are given. Every value between the high and low has an equal chance of being selected. |
| Transparency, Uncertainty | The uncertainly around numbers and how it affects risk ranking is not clear. How do tables of data relate to numbers actually used in risk assessment, especially with respect to uncertainty about numbers, and how this uncertainty affects the risk ranking? | The uncertainty of the estimated number of cases leads to uncertainty of the rank. The principal ranking reported is based on the median number of cases estimated for each food category. |
| Uncertainty, Distribution | How were the most frequent and maximum values selected? | These values were reached by consensus of the risk assessment team and reviewed by the risk manager team, scientific experts, and advisory committees who are knowledgeable of the products. |
| Uncertainty, Distribution | Why were 20% and 50% chosen in the distributions? Why was a uniform distribution used? | The variation in storage times is largely unknown. The uniform uncertainty ranges are based on expert judgment. Uniform uncertainty reflects a state of minimal knowledge. |
| Uncertainty, Distributions | Please explain potential uncertainty introduced by using fitted distributions. | Uncertainty, by definition, attempts to quantify what is not known. It is based on expert judgment (of the risk assessors) of the quality of the available data. Text added to the 2003 risk assessment better describes the process used. |
| Uncertainty, Management | Use uncertainties identified to prioritize new data collection. | The 2001 draft risk assessment was used to determine priorities for the collection of additional data. These new data have been incorporated into the 2003 risk assessment. |
| Uncertainty, Model | The greatest sources of uncertainty are dose response model and virulence of contaminant strains--can be addressed under dose-response and virulence specific sections. | These uncertainties are described in Chapter IV. Hazard Characterization. Sensitivity analyses were not run to determine which uncertainties made the greatest contribution to the final uncertainties in the risks, because the primary objective of the risk assessment was to compare the food categories. Any uncertainty with the dose-response modeling would be equally applicable to all categories. The level of uncertainty was sufficiently low to allow distinguishing pregnancy related and elderly from the total population. |
| Uncertainty, Rank | Large differences in uncertainty resulting from scarce and/or incomplete data over-estimates risk of Listeria monocytogenes and skews the risk ranking | There are large uncertainties associated with the L. monocytogenes concentration characterizations. To some extent, these are represented in the uncertainty analysis. Furthermore, any consistent overestimate in the L. monocytogenes concentrations will be counteracted by the dose-response scaling factor. If the uncertainty is large, then there is a possibility that an extreme is "correct." |
| Variance | Compare the outcomes of a probabilistic risk assessment to the outcomes of risk assessments using interval and/or fuzzy arithmetic to decrease variance due to multiplication. | A comparison of the outcomes of a probabilistic risk assessment to the outcomes of risk assessments utilizing interval and/or fuzzy mathematics to minimize variance is an interesting concept. However, FDA/FSIS utilized the most accepted approach to modeling. Multiplying two databases result in a "real" increase in the width of the distribution. |
| Variance | The variance for product of distributions is larger than the variances of the original distributions. What is the practical consequence? | Distributions increase in width when added or multiplied with other distributions. Since combined distributions do not get smaller, this is a justification for keeping risk assessments as simple as possible. |
| Variation, Distribution | Why a one log uniform variation for maximum growth? Why not a normal distribution? | There were few studies where the maximum growth was clearly determined. Therefore, the minimum knowledge distribution was used. |
| Virulence | Not all strains of Listeria monocytogenes are equally virulent. (Some evidence suggests a frequent finding of low levels of Listeria monocytogenes with strains not connected with human outbreaks. Strains may not be as much a risk factor as found infrequently in large amounts or with more pathogenic strains.) In the absence or virulence markers, it is agreed that one must assume all strains have the same potential for causing illness. Listeria monocytogenes subtypes differ in ability to cause disease. More research is needed on population-based studies, combined with comparative virulence characterization of different Listeria monocytogenes subtypes to ascertain differences in human pathogenicity of subtypes. Studies should include tissue culture models using human and animal cell lines and animal models. Short term changes in risk assessment are not sufficient. | Consideration of strain differences based on best available scientific information was an integral part of the dose-response model. (See Chapter IV. Hazard Characterization, Variability in Virulence section.) |
| Weight | Weighting of studies should not only be based on sample size. Quality of data, method, study design, and representativeness should also be considered | Sample size is a well-established and accepted criterion for weighting. In the 2003 risk assessment, the contamination studies were weighted by sample size, country of origin, and study date. |
| Weight | Why use 54% times 1% instead of 50% times 1% for unpasteurized milk. | 28 out of 52 jurisdictions permit unpasteurized milk, thus, 54%. However, the assessment does round to a figure of 0.5% to calculate raw milk consumption from the total. |
| Weight, Data, Contamination | Giving greater weight to higher percentiles gives more weight to less precise studies or gives undue importance to some data points. | The consequence of not weighting high doses is to generally flatten the curves (i.e., predict higher Listeria monocytogenes levels in samples) because the algorithm is dominated by the greater preponderance of studies at the 0.04 cfu/g level. However, the dose-weighting algorithm is not necessary or used in the procedures employed in this risk assessment to characterized Listeria monocytogenes concentration at retail. Knowledge about the frequency of high levels of Listeria monocytogenes is more uncertain than about the percent positive samples, but these are where the cases of listeriosis come from. This is a tail-driven risk assessment. Further, the approach to modifying contamination was changed to provide more stability, however 100,000 variation iterations and 300 uncertainty iterations were used. The NFPA data did show that high levels of contamination do occur at very low frequency. (See Chapter V. Risk Characterization, Simulation Modeling section.) |