Draft Voluntary National
Retail Food Regulatory Program Standards

Appendix F - Supplement to Standard No. 6 - Compliance and Enforcement

Work Sheet Instructions

This Standard applies to all voluntary and regulatory activities used by a jurisdiction to achieve compliance with regulatory requirements. The desired outcome is an effective compliance and enforcement program that consistently follows through on documented violations and achieves compliance. The sequence and type of follow-up activity a particular jurisdiction elects to use may vary. However, when an out-of-control risk factor or intervention is documented on an inspection report, the expectation is that actions taken to correct the violation will also be documented in the establishment file. For the purposes of self-assessment, follow-up actions have been divided into three types.

• On-site corrective action that occurs at the time of a routinely scheduled inspection,
• Follow-up action that occurs after the routine inspection, such as re-inspections, training, risk control plans, and informal conferences, and
• Enforcement activities such as fines, permit suspension, hearings, mandated training, restriction of operations, embargo, etc.

The measure of success for a compliance and enforcement program under Standard 6 is based on a review of randomly selected establishment files to determine whether documented violations have been resolved satisfactorily in the establishment.

In order to track documented violations through the compliance and enforcement process for a period of time long enough to determine resolution, a fixed point in time must be chosen as the starting point. It is expected that follow-up or subsequent inspections of that facility should show correction of the violations documented at the starting point. The Standard 6 measure uses a concept called the ‘start-point inspection.’

The ‘start-point inspection’ will be the third oldest routine inspection in the establishment’s file if it shows a violation of one of the risk factors or Food Code interventions. If no risk factor or Food Code intervention violation is shown on that inspection, then the fourth oldest routine inspection may be used if it shows a risk factor or Food Code intervention violation. The third oldest routine inspection is determined by starting from the most recent routine inspection in the establishment’s file and working backward chronologically. The fourth oldest routine inspection would be the one prior to the third oldest. If no violation of a risk factor or Food Code intervention is documented on the third or fourth oldest routine inspection, then no ‘start-point inspection’ exists for that establishment.

A sampling of files will be reviewed for compliance and enforcement performance based on the ‘start-point inspection’ concept. The following section provides instructions for the proper construction of a list of sample files and a required alternate list of sample files.

SELECTING THE SAMPLE

The method used to select the sample is to be uniformly applied and must provide a sample that is representative of the total number of retail food establishments in the community. A method of simple random sampling can be used on the principle that all retail food establishments in the inventory are equally exposed to selection.

Jurisdictions with less than 400 total establishments will select at least 20 files for review. Jurisdictions with over 400 establishments will select a sample equal to 5% of the total establishments up to a maximum of 70 files. This initial selection of sample files will be the initial sample and will be the first files reviewed. Sample selection using a table of random numbers or a random number generator is the preferred method of sample selection and can be used with a card file, ledger, list, or automated data system. However, two alternative sample selection techniques acceptable for retail food program self-assessments are presented here.

1. Method 1. The first alternative technique to the use of a random number generator requires that each establishment be identified by a card or strip of paper having the establishment’s name and address, permit number, file number, or other means of positive identification. These identifying cards or slips of paper are thoroughly mixed and the establishment files to be reviewed are drawn one at a time until the required number is obtained.

2. Method 2. The second alternative technique to the use of a random number generator utilizes a card file, ledger, list or data processing record system. When this procedure is used, all the establishments in the program must be subject to sampling. The frequency interval may be determined by dividing the total number of retail food establishments by the number of files needed in the sample. (For example, if there are 800 establishments within the jurisdiction, a sample of 40 would be needed (5% of 80). The frequency interval would be 800 divided by 40, or 20. Thus every 20th establishment shall be selected to make up the initial sample.)  In order to maintain the desired random quality of the sample, the card file, ledger, list or data processing record system should be entered or listed in a random fashion. To establish a starting point when using a frequency interval of 20, write numbers 1 – 20, inclusive, on separate strips of paper and draw one slip at random. The number appearing on that strip of paper represents the first establishment to be drawn. If a ledger or list is being used for sampling and the number drawn is 7, then the seventh entry in the ledger or list would be the first establishment in the sample. The second establishment would be the 27th entry, the third would be the 47th entry and so forth, until the sample of 40 is drawn.

ALTERNATE SAMPLE LIST

Deletion of an establishment from the sample of files to be reviewed will be limited to those establishments which have not been in business long enough to have at least three regularly scheduled inspections or those files where no risk factor or Food Code intervention violation is documented on the ‘start-point inspection.’

When an establishment file is eliminated from the initial random draw, a new establishment file will be drawn from a pre-determined alternate sample list. Alternate files will be drawn in the same manner as the original sample and at the same time as the original sample selection. It is suggested that the number of alternate files selected be at least 30 percent of the original sample size. If a large number of files selected in the initial draw do not have risk factor or Food Code intervention violations on the ‘start-point inspection,’ then a larger alternate sample will be needed.

The sample list of alternate files shall be kept separate from the original sample list. When an original selected file cannot be rated because it has not been in business long enough to have received at least three routine inspections or because it has no risk factor/intervention violation on the start-point inspection, a substitute file from the pre-selected alternate list will be reviewed. Substitute files from the alternate list will be chosen in the order in which the files were draw

If a random number generator or a table of random numbers is used for the initial sample selection, then this same method should be used to select the appropriate number of files for the alternate sample list. Again, this is the easiest and preferred method of sample selection.

If method 1 is used for the random selection, the alternate sample files will be the last files drawn. For example, if the sample size required is 20, then 26 files will be selected, and the last 6 files drawn will be designated as alternative files.

If method 2 is used for the random selection, a separate drawing of the alternate files will be made using an interval determined as follows: the number of establishments in the inventory, minus the number of files drawn for the original sample, divided by the number of alternate files needed. Using our example from method 2 above:

800 (inventory) – 40 (files drawn in the original sample / 12 (30% of the original sample) = 63

To establish a starting point for the new interval of 63, write the numbers 1 – 63 inclusively on separate slips of paper and draw one at random. The number drawn will be the first file selected for the alternate sample and every 63rd file afterward until12 files are drawn.

REVIEWING AND RATING THE FILES

Step 1. Identify the items on the local inspection report that correspond to each of the risk factors and interventions on the worksheet. Record the local item numbers on the "reference key" line of the worksheet. If there is no corresponding local requirement for a particular FDA Code risk factor or intervention, record "NA" for not applicable. You may find the Standard No. 1, Appendix A Worksheets, helpful in making this comparison. Note that the program is not penalized under Standard No. 6 for sections of the Food Code that have not been adopted.

Step 2. Open the first establishment inspection file that was randomly selected in Step 1 above. Identify the third oldest routine inspection report in the file, starting at the current date and working back chronologically. This inspection will be the "start-point inspection" for the review of this file. Using the reference key line on the worksheet, determine which risk factors and interventions were out of compliance at the time of this ‘start-point inspection.’  Place a check under each item that is out of compliance on the horizontal status line. If there is no risk factor/intervention that was out of compliance on the third oldest inspection in the file, you may move to the fourth oldest inspection in the file and use it for the ‘start-point inspection’ if it contains a risk factor/intervention that was out of compliance. If there is no risk factor/intervention that was out of compliance on the third or fourth oldest inspection, eliminate this file from the review and select a substitute file from the alternate list. [NOTE: Be sure to indicate the date of the start-point inspection on the Appendix F worksheet for each reviewed file. This will aid the reviewer during a validation audit.]

Step 3. Review all of the documentation in the establishment file from the start-point inspection forward to the current date and determine whether follow-up action was taken and documented for each of the out-of-compliance risk factors and interventions that were out of compliance on the start-point inspection. Determine whether there was at least one type of follow-up activity for each item that was marked out of compliance. Place "Yes" in the appropriate line and column to indicate that follow-up action was documented in the establishment file. Make a notation below each "Yes" to indicate the type of action taken such as "RH" for Reheat, "WL" for warning letter or "RCP" for risk control plan. If there is no documentation in the establishment file to indicate that follow-up action was taken for each specific risk factor or intervention that was out of compliance, the presumption is that follow up did not occur. Indicate by "yes" or "no" in the last column whether follow-up actions complied with the jurisdiction's written step-by-step procedure for compliance and enforcement.

In order for an individual establishment file to pass, each column marked with a violation at the start-point inspection must have a subsequent "yes" answer to indicate that at least one type of follow-up action was taken. Actions must have complied with the jurisdiction's written step-by-step procedure for compliance and enforcement. A single start-point violation without a final resolution, either correction or a compliance/enforcement activity causes the file to fail. Circle the appropriate "pass" or "fail" notation at the bottom of the work sheet.

Repeat Steps 2 and 3 with each of the randomly selected establishment files. When all of the files have been reviewed, total the number of files that passed and divide by the total number of files that met the sample selection criteria that were reviewed. To meet Standard No. 6, eighty percent (80%) of the files must pass.

See the example following and blank Worksheet.

EXAMPLE:

Sample Work Sheet - Compliance and Enforcement

File No 1

Risk Factors and Food Code Interventions
Establishment Name Seafood Palace	Unsafe Source	Inadequate Cooking	Improper holding Temperatures Hot & Cold	Time/Temperature Parameters not met.(Time as a control, date marking, rapid cooling)	Bare hand contact with ready-to-eat PHF	Poor Personal Hygiene	Contaminated Food Contact Surfaces & Equipment	Consumer Advisory (when required)	Demonstration of Knowledge by PIC	Employee Health Control system or policy implemented.	Was the Written Procedure Followed?
Permit Number 339
Inspection Date (start point) 3 May 2000
Reference Key to local inspection items	1	2, 3, 4, 5	6,7	8, 11	13	14	15	NA	NA	16	Circle One YES or NO
Start Point Inspection Violations		X		X	X	X
Was on site corrective action taken?		Yes RH		YES EM	Yes Glove
Was follow up corrective action taken?				YES RCP	Yes TR
Was enforcement action taken?		YES WL
Each column in which a violation is noted must receive a yes response to one of the three questions in order for the file to pass. Additionally, written procedures must have been followed.										Circle One PASS/FAIL

In this example, the file passes because each of the violations noted on the start point inspection, dated 3 May 2000, has documented follow-up action in the file. The "NA" under Consumer Advisory indicates that the jurisdiction does not have a requirement for this intervention. The "yes" in the last column indicates that the compliance and enforcement procedure of the jurisdiction was followed.

*Define the acronyms and notations used to reflect follow-up action. RH= Reheat to safe temperature, RCP= risk control plan successfully completed, WL= warning letter sent, EM =embargo, TR = training required

Work Sheet - Compliance and Enforcement

File No.

Risk Factors and Food Code Interventions
Establishment Name	Unsafe Source	Inadequate Cooking	Improper holding Temperatures Hot & Cold	Time/Temperature Parameters not met.(Time as a control, date marking, rapid cooling)	Bare hand contact with ready-to-eat PHF	Poor Personal Hygiene	Contaminated Food Contact Surfaces & Equipment	Consumer Advisory (when required)	Demonstration of Knowledge by PIC	Employee Health Control system or policy implemented.	Was the Written Procedure Followed?
Permit Number
Inspection Date (Start Point)
Reference Key to local inspection items											Circle One YES or N0
Start Point Inspection Violations
Was on site corrective action taken?
Was follow up corrective action taken?
Was enforcement action taken?
Each column in which a violation is noted must receive a yes response to one of the three questions in order for the file to pass. Additionally, written procedures must have been followed.										Circle One PASS/FAIL

*Define the acronyms and notations used to reflect follow up action.

---

---

---

---