This Emergency Management Program Evaluation Inspector’s Guide provides detailed information and tools to assist inspectors assigned to evaluate the capabilities and performance of emergency management programs across the DOE Complex. This inspector’s guide is designed specifically as an internal training tool for HS-63 inspectors. HS-63 anticipates making periodic revisions to this guide in response to changes in DOE/NNSA program direction and guidance, insights gained from independent oversight activities, and feedback from DOE Headquarters, field offices, and sites, as well as external stakeholders. Therefore, users of this process guide are invited to submit comments and recommendations to the Director, HS-63, at Steven.Simonson@oa.doe.gov.

Section 1

INTRODUCTION

Purpose

As part of an effort to enhance the appraisal process, the Office of Independent Oversight (HS-60) and the Office of Emergency Management Oversight (HS-63) have prepared a series of documents that collectively provide comprehensive guidance and tools for evaluating emergency management programs across the U.S. Department of Energy/National Nuclear Security Administration (DOE/NNSA) complex. The Independent Oversight Appraisal Process Protocols document describes the philosophy, scope, and general procedures applicable to all independent oversight appraisal activities. The HS-63 Emergency Management Oversight Appraisal Process Guide describes specific procedures used by HS-63 in planning, conducting, and following up emergency management inspections. The information in these appraisal process documents is not repeated here and therefore should be referred to when planning and conducting emergency management program inspections. In particular, the HS-63 Emergency Management Oversight Appraisal Process Guide provides information regarding inspectors’ responsibilities and an overview of inspection activities.

This guide provides a set of detailed tools and references that inspectors can use to plan and conduct an inspection of the emergency management program. These tools serve to promote consistency, ensure thoroughness, and enhance the quality of the inspection process.

Organization of Guide

This section of the guide provides an overview of inspection goals/areas of emphasis.

Section 2 (Program Elements) provides detailed guidance for inspecting emergency management program elements and includes descriptions of various types of data collection activities and the identification of common deficiencies identified in the past.

Section 3 (Analyzing Data and Interpreting Results) contains guidelines on how to analyze information gathered during data collection activities (including the impact that a deficiency in one program element may have on other program elements), interpret the significance of potential deficiencies, assign ratings to the supporting elements of a site’s emergency management program, and identify findings.

Appendix A provides references.

Overview

The tools contained in this guide are intended to be used at the discretion of the inspector. Typically, inspectors select the tools that are applicable and most useful on a facility-specific and inspection-specific basis. Although the guidelines presented here cover a variety of inspection activities, they do not and cannot address all emergency management program variations at all DOE/NNSA facilities. The tools may have to be modified or adapted to meet inspection-specific needs, and in some instances, inspectors may have to design new activities and new tools to collect needed information.

The information in this guide does not repeat all of the detailed information in DOE directives. Rather, it is intended to complement the directives by providing practical guidance for planning, collecting, and analyzing inspection data. Inspectors should refer to this guide as well as DOE directives and other guidance documents at all stages of the inspection process.

One objective in developing HS-63 inspector guides is to provide a repository for the collective knowledge of HS-63’s most experienced inspectors that can be enhanced and updated as inspection methods improve and inspection experience accumulates. Every attempt has been made to develop specific guidelines that are as useful as possible. In addition to guidelines for collecting information, the inspection tools provide aids for prioritizing and selecting activities, then analyzing and interpreting results. The specific guidelines should be viewed as suggestions rather than requirements, and they must be critically examined and interpreted on an inspection-specific basis, taking into account site-specific factors.

Inspection Goals

The primary inspection goal is to determine, with reasonable certainty, whether the emergency management program is both adequately meeting the appropriate standards established by DOE policy and is capable of providing appropriate protection to site personnel and the public in case of an accident at the site. Accomplishing this goal requires that the inspection team determine whether the emergency management program is adequately managed, staffed, trained, equipped, and capable of performing all mission-related tasks and duties.

Compliance/Performance

While an emergency management program inspection includes both compliance and performance activities, a greater emphasis is placed on the performance aspect, as it is more useful in determining whether the emergency response organization (ERO) can perform its mission. Many of the DOE emergency management requirements contained in DOE Order 151.1C are stated in performance terms: that is, they state a capability, duty, or integrated response that must be performed. Therefore, compliance requires effective performance. Even when dealing with policy requirements for which a compliance approach may seem appropriate (e.g., Does the training program contain the required elements?), the HS-63 approach for this topic is to go beyond compliance and determine the performance aspects of these requirements (e.g., Does the training program adequately prepare the emergency director to perform his/her mission?). Therefore, whenever possible, data-collection activities for the emergency management program should be performance-oriented.

Planning Goals

The ultimate goal of planning is to anticipate and provide for every action necessary to conduct the highest quality inspection possible with the resources available. That is an extremely broad goal, and it provides little structure for actual planning. However, it is useful to focus the planning process on several narrower, yet major, goals. Examples of such goals might include:

Understanding the character of the ERO, including its size, composition, and mission; having a general familiarity with how it is trained, managed, and equipped; and understanding the environment in which it operates.
Understanding the facility/site mission and major hazards.
Determining the topics to be inspected and the specific areas of focus for inspection activities.
Determining the specific data collection methods to be used, including any performance tests to be conducted.
Identifying and arranging for the provision of all personnel, administrative, safety, and logistical requirements necessary for data collection.
Producing necessary planning documents.

Planning Decisions

Based on analysis of the information gained from a scoping visit, site document review, discussion with other inspection team members, and discussion with the site points of contact, the inspection team leader must make a number of decisions, including:

Scope and emphasis of inspection activities (including final selection of topics).
Data collection methods to be employed, including performance tests.
Logistics, administrative, and personnel support required, and its sources.
Team members and their data collection activities.
A tentative schedule for data collection activities.

Once these decisions have been made, individual inspection plans can be developed, and the detailed planning of data collection activities can proceed.

Relevant Site Documents

Many of the program elements are based on the site documents listed below. The inspector should review the sections of these documents germane to his/her assigned review area during planning and/or data collection activities. Other relevant documents that are specific to a program element are listed in the appropriate review area of Section 2.

Communications systems testing procedures and maintenance records – procedures and processes for performance tests of notifications systems (e.g., telephone callout, sirens).
Corrective action plans – actions for strengthening previously identified program weaknesses.
Design Based Threat/Vulnerability analysis – provides analyzed scenarios for malevolent acts.
Emergency plan – describes the emergency management administrative program and the site’s overall strategy for responding to emergency events.
Emergency plan implementing procedure(s) and supporting documents – includes implementing procedures, checklists, work instructions, and job aids for responding to emergencies. Implementing procedures will likely include fire department, security department, and some facility procedures. Procedures should address the following topics:
- Categorization and classification, including implementation of the EALs
- Notifications
- ERO recall, activation, and turnover
- Determining protective actions for workers, both involved facility and the site, responders, and the general public
- Implementing protective actions at the site and facility level; including shelter-in-place, accountability, and building/site evacuation
- Consequence assessment processes and tools, including computer manuals
- Software quality assurance (for consequence assessment software)
- Field monitoring and integration
- Incident command, including responses requiring unified command
- Incident response and mitigation
- Operation of the various centers included in the ERO; for example, emergency operations center, technical support center, communications center, and/or operational support center
- Reentry
- Termination and recovery
- Onsite transportation accidents
- Response to events involving OST shipments
- Integrated response to events involving security
Emergency Planning Hazards Assessments (EPHAs) – identify event scenarios and indicators/bases for facility and transportation EALs
Emergency Readiness Assurance Plan (ERAP) – highlights many aspects of the program, including significant changes to the program, assessment schedules, evaluation results and status, performance measures, and program goals, milestones, and objectives.
ERO roster – provides the names of the persons assigned to ERO positions, including alternates.
Exercise evaluations – documentation of recent response performance that identifies program strengths and weaknesses.
Hazards survey – should identify facilities and activities that may incur operational emergencies.
Lessons Learned – a feedback and improvement mechanism that promotes continuous improvement.
Internal or self-assessments - a periodic evaluation of the emergency management program elements performed by the organization responsible for the development and execution of the element evaluated.
Memoranda/letters of agreement or understanding – written agreements among the site, local, tribal, and state jurisdictions. Emergency response commitments and arrangements contained in agreements should be reflected in the emergency plan and implementing procedures.
Transportation EPHA – provides the basis for site transportation EALs. This EPHA may be replaced by the Emergency Response Guidebook.

Section 2A

HAZARDS SURVEYS AND EMERGENCY PLANNING
HAZARDS ASSESSMENTS

General Information

Hazards surveys and emergency planning hazards assessments (EPHAs) form the basis for a site’s emergency management program. The hazards survey serves to establish the “Base Operational Emergency Management Program.” The hazards survey is intended to identify emergency management program needs that are different from those addressed by the EPHA. Therefore, each facility/site should be included in a hazards survey, regardless of the need for an EPHA. The hazards survey contains State, local, and Federal agency requirements related to facility and occupational safety, environmental and effluent controls, and hazardous materials management. It also contains a listing of all hazardous materials and a qualitative evaluation, through a screening process, of whether the hazardous materials will need to be considered in an EPHA. If hazardous materials meet or exceed specified screening threshold criteria, then a quantitative EPHA is performed to determine whether a hazardous material emergency management program is needed. Such a program is needed if the EPHA indicates that release of hazardous materials may result in a classifiable emergency. Other important information derived from the EPHA is the technical bases for determining the size of the emergency planning zone (EPZ), and identification of indications of barrier failures that should be used to develop emergency action levels (EALs).

Because the hazards survey and EPHA(s) form the basis for developing the emergency response program, deficiencies in these documents can lead to deficiencies in other emergency management program elements. The Potential Impacts on Other Program Elements in section 3 of provides a description of these potential impacts.

Relevant Site Documents

In addition to the site documents listed in Section 1, the inspector may choose to review following site documents before or during the performance of onsite data collection.

Authorization basis documents for facilities of interest (Documented Safety Analysis (DSA), Safety Analysis Report (SAR), Facility Hazards Analysis (FHA), and Technical Safety Requirements (TSR)) – used as the reference starting point for postulating potential accidents analyzed in the EPHA.
Chemical, biological, and radiological inventories, documents, or databases – provides data for determining types and quantities of hazardous materials on the site, their location, and for use in the process of developing the hazards survey.
Facility fire plan – provides information on location of hazardous materials within the facility and facility fire suppression systems that may be used to quantify fire scenarios analyzed in the EPHA.
Map of all site facilities – provides the location of all facilities that should be covered by the hazards survey and the relative location of the facilities to the site boundary and public access points.

Data Collection Activities.

Document Reviews

The following data collection activities have been found to be beneficial in evaluating this emergency management program element. The inspector should choose which of the activities to perform based upon the focus of the evaluation and the site-specific features of the emergency management program. Further guidance is found in the inspection plan.

Hazards Survey

A. Inspectors should determine whether a site mechanism is in place that ensures required hazards survey elements are incorporated. The mechanism could be an emergency planning document or procedure. Review the mechanism used to develop the hazards survey and evaluate whether it specifies:

Format and content criteria
A multidisciplinary team appointed to develop the survey and review results
Facility walk downs
Facility personnel participation
A preliminary hazardous material screening process
A three-year review cycle that also ensures new hazards or activities or significant changes to hazardous material inventories are evaluated for emergency preparedness purposes prior to their introduction into the facility.

B. Inspectors should review the hazards survey to determine whether it was prepared in accordance with the site mechanism and possesses all appropriate elements, in matrix or tabular format. In addition, determine whether all facilities, including offsite hazardous facilities (e.g., water treatment plant using chlorine) and activities (e.g., transportation artery) that may impact the site are considered in the hazards survey.

C. Inspectors should review the hazards survey to determine whether hazardous material screening, if performed as part of the survey process, is consistent with DOE Order 151.1C and DOE Guide 151.1-2, Technical Planning Basis. The following activities should be performed to support this evaluation:

Perform a comparison of threshold quantities of hazardous materials to site quantities listed in the hazards survey or referenced databases.
Determine whether screening is conservative and consistent with the screening criteria set forth in DOE Order 151.1C.

D. Inspectors should cross-check emergency plan facility descriptions, the site map, and the listing of facilities in the hazards survey to determine whether the hazards survey includes all facilities.

E. Inspectors should review the site’s hazardous material database(s) to determine whether all hazardous materials are listed in the hazards survey, or by reference, and have been identified.

F. Inspectors should walk down a number of facilities, including the shipping and receiving warehouse, and observe transportation activities to check the accuracy of chemical, biological, and radiological hazardous material inventory database(s) used as the basis for determining amounts of hazardous materials that are used or stored in facilities.

G. Inspectors should walk down the emergency response facilities to determine whether an up-to-date copy of the hazards survey is available in each facility.

Emergency Planning Hazards Assessments

H. Inspectors should determine whether a site mechanism is in place that addresses EPHA development and maintenance. The mechanism could be an emergency planning document or procedure. Review the mechanism to determine whether it specifies:

Format and content criteria
A process for preparation, review, and approval
Facility personnel participation
A multidisciplinary team appointed to prepare and review results
A three-year review cycle that also establishes a mechanism to ensure that new hazards or activities or significant changes to hazardous material inventories are evaluated for emergency preparedness purposes prior to their introduction into the facility.

I. Inspectors should review the EPHA(s) to determine whether they were prepared in accordance with the mechanism.

J. Inspectors should review the EPHA(s) to ensure an EPHA has been prepared for each facility containing hazardous materials that were not screened out. Determine whether hazardous materials exceeding screening criteria are fully characterized (amount, location, condition of use, material properties, controls, etc.) to support development of scenarios and analysis of possible releases.

K. Inspectors should review applicable EPHA(s), Design/Safety Basis Documents, and Vulnerability Analysis, if applicable, to determine whether the full spectrum of emergency events and conditions that could cause releases of hazardous material are analyzed. For example, determine whether:

Events include low-probability/high-consequence events as well as high-probability/low-consequence events.
Vulnerability Analysis, Design Basis Threat analysis, DSA/SAR, and/or FHA scenarios are included (either directly or by reference) in the EPHA(s).
Accident initiators, such as corrosion, manufacturing defects, malfunctioning equipment or control systems, and procedural or human error, are addressed.
External causes, such as impacts of natural phenomena, accidents at nearby facilities, vehicle and/or aircraft crashes, and minor, moderate, and extreme malevolent acts, are included.
Pathways for waterborne and ground releases, as well as airborne release paths, are considered if the pathway requires time-urgent response.

L. Inspectors should review the EPHA(s) to determine whether barriers to release of hazardous materials are identified, together with the possible initiating events, accident mechanisms, and equipment failures. In addition, determine whether indicators (alarms, instrument readings) of barrier failures for use in developing EALs are identified and whether areas where indications are lacking have been identified and action initiated (e.g., plant modification requests) to correct the condition.

M. Inspectors should review the EPHA(s) to determine whether facility and site boundaries have been correctly defined and whether meteorological assumptions are correct.

N. Inspectors should perform an independent consequence assessment analysis on several scenarios to ensure that the EPHA(s) results are accurate and reproducible.

O. Inspectors should review the EPHA(s) dispersion models for determining consequences to determine whether they are equivalent to emergency response models used by the ERO.

P. Inspectors should review the EPHA(s) to determine whether correct protective action criteria are applied to potential release scenarios (e.g., Was the correct hierarchy of determining Acute Exposure Guideline Level (AEGL), Emergency Response Planning Guideline (ERPG), or Temporary Emergency Exposure Limit (TEEL) values used? Was uranium considered as a toxicological hazard as well as a radiological hazard?).

Q. Inspectors should collaborate with the Readiness Assurance inspector to confirm that the DOE/NNSA field office operations manager (designated representative) reviewed and approved the hazards survey and EPHA(s). In addition, determine whether the DOE/NNSA field office operations manager reviewed, approved, and submitted the site EPZ to appropriate Headquarters staff. Confirm that offsite jurisdictions concur with the EPZ determination.

R. Inspectors should walk down the site’s emergency response centers to determine whether the consequences to onsite and offsite receptors of interest have been calculated and documented in readily available documentation.

S. Inspectors should observe and evaluate the use and usability of EPHA(s) by consequence assessment personnel in the limited scope performance tests.

Common Deficiencies/Potential Concerns

This section identifies areas where concerns or deficiencies have been identified in previous inspections. By reviewing this information before gathering data, inspectors can be aware of these deficiencies and concerns during interviews, walkdowns, and other data gathering activities. However, the inspector should not simply focus his/her evaluation on whether these deficiencies exist at the site being evaluated, but rather should consider all aspects of this emergency management program element (including strengths and weaknesses).

No mechanisms for preparing, reviewing, and approving hazards surveys and EPHAs are established.
The role of facility personnel in the hazards survey and EPHA development and maintenance process is not defined.
The hazardous material screening process used is not documented.
Mechanisms for updating hazards surveys and EPHAs prior to introducing new hazardous materials or activities, or when changes to hazardous material inventories occur are not established.
The required hierarchy of protective action criteria used in developing consequence assessments is not used.
The site’s emergency management program and other programs, such as that for developing/maintaining Authorization Basis documents, is not effectively integrated.
The impacts of wildland fires, particularly in areas with contaminated land, are not considered in the EPHAs.
The EPZ is not approved by the local DOE/NNSA operations office manager and/or is not coordinated with offsite jurisdictions.
HS/EPHA authors do not follow site procedures when developing the documents.
Up-to-date Acute Exposure Guideline Level-2 (AEGL-2), Emergency Response Planning Guideline-2 (ERPG-2), or Temporary Emergency Exposure Limit-2 (TEEL-2) values are not used.
A full spectrum of potential emergency scenarios, including: malevolent act and transportation event scenarios; low- probability/high-consequence (i.e., beyond design basis) accident events as well as high-probability/low-consequence events are not analyzed.
Accurate material-at-risk amounts are not used in consequence assessment calculations that are consistent with transport, storage, and operational practices.
Consequences at all significant receptor locations are not assessed.
Offsite hazardous operations that could impact the site, including fixed facilities and transportation activities, are not evaluated.
The results contained in the EPHAs as the basis for categorization/ classification methodology (e.g., using indications of potential barrier failures and distance to protective action criteria to develop EALs) are not used.
Results contained in the hazards surveys and EPHAs is not used to serve as the basis for medical responses.
The EPHAs results (distances to thresholds for early lethality and protective action criteria) to determine EPZ configuration are not used.
Individuals do not have sufficient emergency management expertise to effectively evaluate and approve hazards surveys and EPHAs.
Mechanisms for notifying the emergency management organization of changes in hazardous material inventories and facility operations that may affect the facility EPHA are not used.
Accurate inventory information with such methods as facility walk downs are not used.

Section 2B

PLANS AND PROCEDURES

General Information

The emergency plan documents the emergency management program and describes both its administrative and response aspects. The plan is executed through a series of implementing procedures, often called emergency plan implementing procedures, which contain the appropriate actions for both the Cognizant Field Office and contractor in response to an event. At a minimum, the emergency plan will include the requirements of an Operational Emergency Base Program, which is based on an appropriate hazards survey and screening process and provides the framework for response to serious events as discussed in section 2A. Biosafety facilities must have a designated Responsible Official and incident response plans that are integrated with the site’s emergency plan. Sites with leased facilities must have documented arrangements that integrate the leased facility into the site program; including provisions for control and reporting of hazardous materials.

The administrative program established by the emergency plan and implementing procedures should include processes for the development, review, approval, and periodic update of plans and procedures; including supporting checklists and operator aids. A document control system should ensure that procedures are available for use at all the response locations, and are the currently approved versions. In addition, procedures and processes must be in place so that memoranda of understanding (MOUs) and agreement (MOAs) or mutual aid agreements with local, state, tribal, and Federal agencies are developed, approved and maintained

Relevant Site Documents

The inspector may choose to review any of the site documents listed in Section 1 before or during the performance of onsite data collection. It is important to note that the following plans and procedures are typically evaluated by the specified functional area evaluator when assigned.

Inspection Element	Evaluator

Hazards survey/EPHA process procedures	Hazards survey/EPHA evaluator
Offsite Response Interface agreements	EPI Evaluator
Training plans	Training, Drill, and Exercise evaluator
EPI Plans and Procedures	EPI evaluator
Corrective action/Issues management procedure	Readiness Assurance evaluator

Data Collection Activities

Document Reviews

A. Inspectors should evaluate the emergency management Program Administration attributes by reviewing the emergency management plan(s) and implementing procedures to determine whether:

The plan describes the site’s emergency management organization, including the Cognizant DOE Field Office, and roles and responsibilities for planning and preparedness.
Roles, responsibilities, and requirements for program administration are clearly established including those for:
- Contractor and DOE program administrators
- Functional area managers (for example, radiological controls manager)
- Line managers (for example, operations managers and facility managers)
- Responsible official for biosafety facilities
- Leased facilities
The plan describes the emergency response organization; including individual positions, operations, and interfaces.
The plan effectively integrates Federal, state, tribal and local authorities (including mutual aid and support) and conforms to the requirements of the National Response Plan (or successor document) and National Incident Management System.
The plan and implementing procedures provide for an adequate document control system; processes for maintaining and updating the plan and procedures are documented and effective.
Memoranda of agreement/understanding and mutual aid agreements are current and commitments made to offsite officials are maintained.
The plan describes the site’s overall administrative program, including:
- Hazards surveys and EPHAs
- Training and drills
- Exercises
- Readiness Assurance

Note: These program areas are generally inspected as separate, stand-alone areas and the inspector assigned to plans and procedures will not be directly involved in their inspection, but should verify that they are addressed in the emergency plan.

B. Inspectors should evaluate if the emergency plan describes the ERO in sufficient detail to allow ERO personnel to understand and execute their integrated roles and responsibilities during an emergency, appropriately advise offsite authorities of the site’s emergency response, and enable responsible individuals to develop the emergency plan implementing procedures necessary for effective execution of the plan at all levels and locations. To evaluate this review the emergency management plan and implementing procedures to determine whether:

The management structure is adequately described, including the:
- Division of authority and responsibility (chain of command) among the ERO participants; for example, the incident commander and the emergency director.
- Clear progression of decision making authority, including turnover of responsibility, as an event progresses and successive organizations are recalled and operationally staffed.
- Identification of a single individual with responsibility for the overall response (based on actual or potential emergency conditions).
- Roles, responsibilities, and authorities of DOE personnel, and their integration into the overall ERO structure.
Each subordinate organization/venue (for example, incident command post, communications center, or emergency operations center) and each ERO position has its functions, authorities, and responsibilities documented.
The size, complexity, and response capabilities of the ERO structure are consistent with the hazards at the site and its facilities. For example, if radiation is a hazard, the ERO should have radiological decontamination and field monitoring capabilities.
The relationship of the ERO with other onsite response elements, DOE/NNSA field/operations office, DOE Headquarters, and offsite response agencies/organizations (for example, state, local, tribal) is identified and implemented.

C. Inspectors should review the plan and procedures for initial actions and ERO activation to determine whether:

Building, facility or activity procedures provide for timely identification of potential or ongoing emergencies and transition to facility/site emergency response procedures.
Initial actions by on-shift staff, such as categorization and classification and offsite notifications, are described and are supported by an effective set of procedures and checklists for initiating ERO activation and initial response actions.

D. Inspectors should review plans and procedures for ERO operations to verify that:

Interfaces are clearly established and mechanisms for assigning tasks and priorities are in place.
Internal and external communications interfaces and authorities are identified.
Supporting tools and equipment are addressed; including operating procedures or work instructions, when appropriate.
Mechanisms for transition and turnover of command are established.

E. Inspectors should review plans and procedures governing roles, responsibilities, and operations for some or all of the following: incident command post, emergency operations center, control center, technical support center, field monitoring teams, security, and fire and rescue.

F. Inspectors should evaluate the categorization and classification functions by reviewing the emergency plan and associated implementing procedures to determine whether:

The position(s) responsible for categorizing and classifying an event is/are clearly identified in the plan and that implementing procedures unambiguously execute these responsibilities.
The responsible individual is available to perform the action in a timely manner commensurate with the hazard and the proximity to affected populations.
Transition and turnover of this responsibility is clearly established in plans and implementing procedures.
Criteria for categorizing operational emergencies are clear and unambiguous to the decision maker; are stated in terms of readily available indications or observable conditions; and are based on protective action criteria thresholds in accordance with DOE Order 151.1 C.
A tool (i.e., an EAL-like tool) for recognizing and categorizing biological events, based on recognition factors identified in the EPHA, is part of the emergency management program plans and procedures for biosafety facilities, and includes default initial protective actions.
Thresholds for recognizing emergency conditions are integrated to the extent possible with facility operational procedures; including routine monitoring, alarm response, abnormal, and emergency procedures.

G. Inspectors should evaluate the EALs and applicable classification implementing procedure(s), work instructions, and job aids to determine whether:

Classification can reasonably be accomplished promptly and accurately using a current set of a facility/site-specific EALs and, for transportation events, the use of the Emergency Response Guidebook..
Facility/site-specific EALs are developed and approved for the spectrum of events (in the hazards analysis) on the basis of measured or predicted hazardous material consequences at specific receptor locations (i.e., facility and site boundaries).
EALs for classifying Operational Emergencies are clear and usable, enable timely implementation by the decision maker, and are comprehensive and anticipatory of potential/future consequences.
EALs are stated in terms of readily available indications or observable conditions, and symptomatic EALs are used whenever possible for event recognition.
Implementing procedures accurately apply the EALs that are derived in the EPHAs and any supplemental EAL document or manual. (Note: Coordinate this activity with the inspector of hazards surveys and EPHAs.)
Classification of an event leads to a set of pre-determined onsite protective actions and offsite protective action recommendations.
Procedures direct continuous monitoring and reassessment of event status, and modification of the classification for event degradation or indication the event is more severe than originally perceived.
Sitewide EALs have been prepared for incidents; such as, terrorist threats, earthquakes, tornados, fires, and other external events, which affect multiple facilities.
Procedures direct the involvement of users to develop or validate the usability of EALs.

H. Inspectors should evaluate the notifications and communications function by reviewing the emergency plan and implementing procedures to determine whether:

The communication plan and supporting systems are detailed in the plan and procedures, and support timely notifications, as required in the DOE Order 151.1C.
The site plan and procedures establish a formal communications system to report emergency information and make notifications to: facility workers, co-located site workers, onsite and offsite initial responders, ERO, and public present onsite.
A rapid notification and recall system is used to make initial and follow-up notifications to primary and alternate staff. The system and procedures provide for authentication and feedback indicating successful contact.
Pre-arranged, standardized formats for onsite and offsite notifications and public address announcements (implementing protective actions) are available for various emergency scenarios and classifications.
The processes and communication system to achieve evacuation, assembly, and personnel accountability are identified and are adequate.
Effective communications planning and procedures support information flow within the ERO and effective command and control during an emergency.
Twenty-four-hour notification points-of-contact lists are maintained current and are readily available.
Appropriate record-keeping is implemented by the plan and procedures to enable reconstruction of the progression of event activities.
Notification processes and supporting systems are designed to meet timeliness requirements; for example the system supports simultaneous notifications to multiple locations.
A message verification process is included.
The positions responsible for initiating and receiving notifications to offsite agencies/organizations are identified.
Emergency managers are responsible for reviewing and approving notification forms containing event classification information before they are disseminated to offsite authorities.
Follow-up notifications are required periodically or when significant changes occur.
A system is required for formal documentation of notifications made. (The evaluator should keep in mind the potential for legal review and litigation that may follow an emergency event that requires precise record keeping.).
The use of language that is understood by recipients (for example; identification of zones or sectors for protective action recommendations) is required.
Security provisions are commensurate with the type of information to be transmitted.
The notification forms contain the standardized, specific critical information specified in the Order (see for example, DOE G 151.1-4, Response Elements, Section 5, Appendix C).
Notification forms reflect notification procedure requirements and are consistent with agreement documents.
Provisions are established to review notification messages for classified information or for unclassified controlled nuclear information.

I. Inspectors should evaluate the consequence assessment function by reviewing the emergency plan, implementing procedures, and other emergency management supporting documents to determine whether:

Provisions are made to perform a timely initial assessment of the potential/actual release using available real-time data; in particular, current meteorological data.
In-depth assessments are performed throughout the event; and are integrated with categorization, classification, protective action and reentry processes.
A procedure and supporting processes are in place to utilize NARAC, if required.
Procedures and processes are in place to obtain additional resources and integrate activities, as necessary, to perform field assessments of the extent of releases, both on site and offsite.
Provisions have been established to obtain the assistance of DOE/NNSA emergency response assets; such as, the Radiological Assistance Program, the Accident Response Group, and the Aerial Measuring System, to support consequence assessment, if necessary.
Inspectors should determine whether the procedures, processes, and supporting tools for timely, accurate consequence assessment during an event are in place. For example,
- Source term information is identified for the range of potential emergency events and conditions analyzed in the EPHAs, and the information is in a form (e.g., tabulated) for ready reference and access.
- Procedures are established for incorporating event-specific data (e.g., source term, meteorology, receptor locations) into consequence analyses, and integrating available data from installed instrumentation for both on site and offsite field measurements.
- Procedures identify and reference data sources, such as instruments or documents, which are to be used to determine potential source terms.
- Standard protocols for communicating consequence assessment data and results have been established to minimize the potential for errors in interpretation.

J. Inspectors should evaluate the field monitoring component of consequence assessment by reviewing the emergency plan and implementing procedures to verify that the facility/site has processes in place to implement field monitoring and integrate the results into the consequence assessments, including:

Adequate planning exists to safely dispatch field teams during an event.
Procedures, processes, and supporting data collection devices (e.g. instruments and forms) are available to support timely collection, transmission, and use of field data for chemical, radiological and biological releases.
Procedures and processes are in place to coordinate offsite monitoring activities with responsible offsite authorities, to integrate the results of monitoring with ongoing consequence assessment activities, and to share/compare results of subsequent consequence assessment results.
Standard protocols for communicating monitoring data and results have been established to minimize the potential for errors in interpretation.

K. Inspectors should review quality assurance processes to verify that dispersion model software, including computational software and supporting databases, contain accurate information.

L. Inspectors should evaluate the protective actions and reentry function by reviewing the sitewide emergency plan, building-specific emergency plans, and implementing procedures to ensure:

Adequate planning and coordination with site support groups, such as security and transportation, to facilitate implementation of the protective actions.
Building/facility emergency plans are in place to support building evacuation, assembly, accountability, shelter-in-place, etc.
Plans, procedures and processes support building, facility, or site evacuation; including evacuation routes, assembly/shelter areas, integration of site support assets, etc. Ensure there are provisions to enable personnel with disabilities to implement protective actions.
Plans and procedures implement timely, accurate accountability.
Building/facility procedures address the actions necessary to implement effective shelter-in-place for the expected emergencies.
Plans, procedures, and equipment to periodically verify habitability of buildings and facilities, including emergency facilities, are in place.
Protective actions for on-scene responders and rescue teams (for example) are formulated and implemented; including measures to ensure that security, fire, medical, and other response personnel are protected from exposure to hazards during the course of their movements while supporting response. Plans and procedures should address the use of emergency exposure limits and turn back limits.
Site procedures provide for reentry and/or search and rescue:
- Safety and health planning is conducted prior to conducting search and rescue, including provision for a rescue team for the reentry team. The use of the buddy system, methods to determine appropriate personnel protective equipment, and a process to account for search and rescue personnel should be stipulated.
- Search and rescue team composition is predetermined, and team members are qualified to perform the task.
Responsibility to authorize response personnel to receive exposures in excess of site limits is documented.
Processes for authorizing and documenting emergency worker exposures, including emergency exposures are in place.

M. Inspectors should evaluate the emergency medical support function by reviewing the emergency plan and implementing procedures for emergency medical support to verify:

The number of personnel, organizations, or offsite assets responsible for emergency medical services is identified and treatment protocols coordinated.
If medical support organizations are offsite, a mutual aid agreement (or equivalent) is in place.
Procedures for treating contaminated, injured personnel are in place, both onsite and offsite.
Planning and procedures for a mass casualty situation are in place.
Procedures for notification and coordination for releases from biological facilities are ready.
Biosafety surveillance plans for detecting unusual medical events are established onsite and specific responsibilities for surveillance and reporting are identified.
Procedures and/or protocols for distribution and use of prophylactic drugs are in place, when appropriate.

N. Inspectors should evaluate the termination and recovery function by reviewing the emergency plan and implementing procedures to ensure:

The decision to terminate emergency response is made by the site ERO and coordinated with all principle participating response organizations (i.e., local, state, tribal, DOE Headquarters, and other participating Federal agencies).
An approved, pre-determined set of criteria for terminating a classified emergency [e.g., an airborne release of (or loss of control over) hazardous material] is met. Selected termination criteria follow those provided in the Emergency Management Guide.
Termination activities provide for the establishment of a recovery organization and a recovery plan, as necessary to address the follow-up from the event.

Facility/Equipment Walkdowns

A. Inspectors should perform a walkdown of site facilities to determine whether:

Indicators referenced in the EALs are available, correctly identified, and useful for classifying events (e.g., readable, correct range). Also note any available indications that could have been incorporated into EAL threshold indicators.
Plans, procedures, checklists, work instructions and other supporting documents are available, current, and appropriately controlled in applicable facilities and control centers.
Control centers and areas such as the incident commander’s command vehicle contain the procedures and tools necessary to formulate and implement timely protective actions for all affected population groups.
Buildings have an appropriate set of alarms, both local and general, and public address systems. Note the location of area alarms and public address systems, and how and where they are activated. Particular attention should be made to notification methods in areas of high noise, where deaf employees work, or in remote areas.

B. Inspectors should walk down locations where consequence assessments and protective actions are formulated to determine whether:

Tools for performing timely initial consequence assessment are available as appropriate.
Consequence assessment hardware and software is operable in accordance with procedures and operators guides.
Necessary presentation materials, such as maps and data recording sheets, are available.
Necessary references are available; such as, conversion factors, protective action criteria, EPHAs, EALs, material safety data sheets, National Institute for Occupational Safety and Health handbooks, DOT Emergency Response Guidebook, and meteorological data.

C. Inspectors should walk down the facilities where notifications are made to determine whether:

A points-of-contact list is available and current.
Correct notification forms are available and conform to the notification procedure.
The rapid notification and recall system is able to make initial and follow-up notifications to primary and alternate staff as specified by procedures.

Interviews

A. Inspectors should interview emergency management organization personnel responsible for maintaining the emergency plan and implementing procedures to evaluate:

The document control system for preparing, reviewing, revising and issuing the plan and procedures.
The system for preparing, reviewing, revising and issuing work instructions, checklists, and operator aids.
The processes for ensuring that documents available to the ERO are kept current, including recall lists and computer based notification systems.

B. Inspectors should interview site personnel responsible for developing EALs to determine whether areas where the unescorted public are allowed can be evacuated within one hour of an emergency event and whether those areas are considered within the site boundary for emergency classification purposes.

C. Inspectors should interview onsite emergency managers to determine whether they have a clear understanding of their responsibilities regarding determining and issuing protective actions for site personnel and protective action recommendations to state and local government organizations. Determine emergency managers’ awareness of employees with any handicap, particularly deaf employees and those with mobility problems. Interview the emergency response managers and initial decision makers responsible for classification to determine whether they are active participants in development, verification, or validation of EALs. Inspectors should interview initial decision-makers to assess whether they can interpret initial assessment tools to refine pre-determined protective-action decision-making.

D. Inspectors should interview emergency planners to determine whether:

Provisions have been established for continuous monitoring of critical parameters that provide information needed to continually assess the consequences of an event.
A method has been established for sharing and comparing results and resolving differences among different response organizations.
The format, content, and level of detail of consequence assessment projections or measurements will support public information activities.
Mechanisms are available to assure quality control of tools used in consequence assessment, such as meteorological system and dispersion analysis hardware and software.
An evaluation of the consistency of calculation results among the EPHA scenarios, the ERO calculational methods, and DOE Headquarters and state/local agencies’ results for selected scenarios has been performed. Determine whether any differences have been rationalized and documented.

E. Inspectors should interview onsite personnel responsible for notifications and communications to determine whether the understanding of the individual(s) responsible for the notifications and communications is consistent with the plan and procedures. (Interviews may reveal processes and procedures that have not been included in the emergency plan).

F. Inspectors should interview the manager of the site’s emergency planning department to determine whether:

Organizations receiving emergency notifications agree to the contents of the message notification form, use the same form as the site to minimize errors, and have the capability to receive and review reports on a 24-hour basis.
Development of the form was coordinated with, and agreed upon by, offsite officials.
The forms or lists are formally controlled and are routinely updated and validated.

Common Deficiencies/Potential Concerns

This section identifies areas where concerns or deficiencies have been identified in previous inspections. However, the inspector should not simply focus the evaluation on whether these deficiencies exist at the site being evaluated, but should consider all aspects of the emergency management program element (including strengths and weaknesses). By reviewing this information before gathering data, inspectors can be aware of these deficiencies and concerns during interviews, walkdowns, and other data gathering activities.

ERO

Roles and responsibilities in the ERO chain of command (for example, between the emergency director and the incident commander) are not fully documented in plans and procedures, especially for security-related events.
ERO roles and responsibilities during off-normal hours are not clearly defined.
On-scene command roles and responsibilities under unified command are not clearly defined.
The command structure and roles and responsibilities for an event involving an Office of Secure Transportation shipment on a DOE/NNSA site are not clearly defined, where applicable.
ERO staffing and organization are not analyzed and procedures revised or updated when changes occur in responsibilities, such as performing the initial notifications.

Protective Action and Reentry

Tools to determine how far and wide an area downwind to apply protective actions are not included with procedures.
On-scene protective action tools are not consistent with EOC protective-action tools (e.g., sector zones versus building numbers) resulting in delayed actions or misunderstandings in protective actions directed and protective actions taken.
Procedures do not consider building exchange rates with outside air or specify tasks, such as closing doors and windows and securing ventilation systems, when sheltering in place.
Procedures do not fully consider plume versus puff release consequences or hazardous material transport times when formulating protective actions.
Procedures for updating protective action recommendations following analysis of consequence assessment or field monitoring data are not available.
Procedures are not supported by tools for overlaying consequence projections onto maps showing impacted areas, receptors, and applicable protective actions.
Emergency plans and procedures lack guidance on roles and responsibilities for protecting deployed protective forces during an event.

Section 2C

OFFSITE RESPONSE INTERFACES

General Information

Pre-planning is necessary to identify the interfaces for offsite organizations that are responsible for protecting the public or that will respond during an emergency. This includes officials from local, state, tribal and Federal agencies who are responsible for assessing offsite protective action recommendations and implementing appropriate protective actions. This element also includes the pre-planning necessary to obtain and provide mutual support during an event. It includes arrangements with offsite agencies such as: local and state fire and law enforcement agencies, local hospitals and ambulance services, county or state public health officials, local and state emergency management officials, and Federal agencies (Federal Bureau of Investigation). Plans should identify these interfaces and their expected role in an emergency, procedures should provide for their integration with the ERO, and agreements should be in place to ensure that roles and responsibilities are understood and accepted.

Interfaces with offsite response authorities and organizations are an integral part of the DOE/NNSA Comprehensive Emergency Management Program.  DOE Order 151.1C requires that effective interfaces be established and maintained to ensure that emergency response
activities are integrated and coordinated with the Federal, Tribal, State, and local agencies and organizations responsible for emergency response and protection of the workers, public, and environment and that they be in accordance with the requirements of the
National Response Plan (NRP) and National Incident Management System (NIMS).

This section provides guidance for inspectors reviewing this aspect of the emergency management program. Specific areas of significance include the provision of:

Appropriate identification of offsite organizations based on the response needs identified in the EPHA;
Detailed programmatic goals and objectives that reflect an ongoing integrated relationship with State, local, and tribal governments before and during an emergency;
Memoranda of Understanding exist that are current and accurately reflect the interrelationship and planning and response expectations between onsite and offsite officials

The inspector assigned this area should coordinate and collaborate with the inspector responsible for Plans and Procedures and Emergency Public Information to effectively and efficiently execute the inspection plan.

Relevant Site Documents

In addition to the site documents listed in Section 1, the inspector may choose to review following site documents before or during the performance of onsite data collection.

DOE/NNSA radiological emergency response asset documents –provides details to support offsite officials in the event of a radiological incident. Facilities/sites should coordinate with offsite officials to provide information on the availability and capabilities of DOE/NNSA radiological emergency response assets; and how to access and utilize these Federal Assets.
Exercise package – describes the level of participation of offsite responders.
Training documents - details the availability of annual training to offsite entities and responders.

Data Collection Activities

Document Review

A. Inspectors should evaluate the offsite response interface function by reviewing the plan and implementing procedures to ensure:

Interfaces with offsite personnel responsible to protect the public and environment are described in the site emergency plan and implementing procedures.
Written agreements are in place for emergency services identified in the hazards survey/EPHAs. Agreements should be in place as a commitment to support and provide protocols to obtain agreed upon services. These may include the following.
- Offsite emergency medical responses for events with mass causalities are specified.
- Hospital emergency rooms, including treatment of contaminated injured personnel, are addressed.
- State and local governments and tribal nation communication interfaces established.
- Mutual aid from local and state fire and police departments, including periodic training to familiarize them with site hazards is established.
- Coordination of offsite field monitoring activities is considered.
- Protocols to activate Radiological Assistance Program teams are specified.
- Onsite positions authorized to request aid from the offsite agency are identified.
- Site personnel have contact information for Offsite individuals authorized to implement site requests.
- Agreements establish specific responsibilities, authorities, and command structure to unify site and offsite responses effectively.
- Public information release protocols, including privacy information protocols are specified.
- Financial arrangements, including commitments by the site to provide training, equipment, and facilities to the offsite provider and indemnification for injury to persons for loss and damage to property are stated.
- Periodic re-examination of the provisions and a renewal or termination of agreements are specified.
- Agreements are signed by authorized individuals representing the site and offsite organizations.
- If a site provides support to an offsite agency under the “good neighbor” policy, those support interfaces should be documented.
- A means to attain an explosive ordinance demolition team is provided.

B. The inspector should review agreement documents to identify site commitments and then ensure there are implementing mechanisms for them.

C. The inspector should ensure that the site provides training to offsite entities within the emergency planning zone by reviewing training materials to ensure information is provided in the following areas:

Site descriptions and operations
Site hazards
Emergency Plans and procedures
Event classifications and protective action recommendations
Public warning mechanisms
Consequence assessment documents
News release protocols
Media Center/JIC Operations and locations
Public Affair contact information
Drill and exercise invitations
Decontamination procedures, if necessary
Use of prophylactics, if necessary

D. The inspector should ensure that the site is involved with the Local Emergency Planning Commission (LEPC) through review of charters and meeting minutes.

Interviews

A. Interview contractor and DOE/NNSA site office personnel responsible for establishing and maintaining interfaces with offsite authorities to determine the extent of the program (such as the level of participation with the LEPC), the program status, and whether it is consistent with written descriptions.

B. Contact the appropriate local, state, and tribal authorities to whether they have the current implementing documents, understand the site emergency response protocols and any mutual aid agreements, and are satisfied with the level of support and cooperation from the site.

Common Deficiencies/Potential Concerns

This section identifies areas of concern or deficiencies identified during previous inspections. By reviewing this information before gathering data, inspectors can be aware of these deficiencies and concerns during interviews, walkdowns, and other data gathering activities. However, the inspector should not simply focus his/her evaluation on whether these deficiencies exist at the site being evaluated, but rather should consider all aspects of this emergency management program element including strengths and weaknesses.

The program lacks clearly defined and integrated responsibilities among and between various site and DOE/NNSA organizations for offsite response interface.
Site individuals establishing and maintaining interface agreements do not have appropriate authority, knowledge, and training to be effective.
Agreement documents are not current or do not appropriately reflect the interrelationship and expectations between onsite and offsite officials.

General Information

The training, drills, and exercise element of the emergency management program includes the process used to determine what training is necessary for the ERO, the presentation of that training, and the processes used to measure the effectiveness of training to ensure that all ERO personnel are capable of performing assigned tasks. The hazards surveys and EPHAs provide a basis for the type of training that is needed. In addition, the site needs to consider what training is needed for offsite personnel (e.g., medical personnel and fire fighters) who may support the site during an emergency. The outcome of the training, drills, and exercise program should be ERO personnel who are capable of performing their emergency response duties. Typically, the evaluation of the training program for EPI personnel and offsite response interface personnel is performed by the EPI inspector and the training program for the federal ERO members is evaluated by the Readiness Assurance inspector.

The exercise element of the emergency management program is used to test all other program elements. It is used to validate program components or identify weaknesses for correction to continuously drive program improvements. It is similar to a drill in that it is a hands-on demonstration of response to a simulated emergency but differs significantly in conduct. Exercise conduct prohibits many of the teaching aspects of a drill, such as coaching and correcting inappropriate performance while in play, and unsatisfactory performance of evaluated objectives results in findings. Disposition of exercise findings should be executed through a formal issues management program. The effectiveness of the issues management program is typically evaluated by the inspector assigned to the Readiness Assurance functional area.

Relevant Site Documents

In addition to the site documents listed in Section 1, the inspector may choose to review the following site documents before or during the performance of onsite data collection.

Training plan or administration document – provides the administration requirements for the training of each team within the ERO (EOC cadre, consequence assessment team, etc.); provides the frequency of the drills; sets criteria for successful completion of training; describes the annual refresher training; specifies training waiver/equivalency policy; provides for the administrative requirements to provide training and record the results; and details the instructor requirements.
Training schedules – provide the site annual training schedule for training classes, exercises, and drills. Review prior two years and current year schedules.
Training, drill, and exercise status reports – provide records of completed training activities and gives insight to a site’s ability to maintain their training schedule.
Training records – document the training completed for each person assigned to an ERO position.
Required reading program – provides applicable reading assignments and documents completed training. It could take the form of paper records, e-mail, or be web-based.
Annual refresher training - provides periodic, new, reinforcement, or lessons-learned training.
Lessons-learned program – provides emergency management related lessons learned from a global perspective (complex and industry wide). Typically managed by a site lessons-learned coordinator. Lessons learned should also be derived from site drills, exercises, and actual events.
Qualification criteria – provide the set of training, experience, and evaluation needed to staff certain ERO positions.
Training courseware – includes instructor guides and notes, lesson plans, student handouts, learning objectives, tests, computer-based training, on-the-job training documents, and other training-related materials used to present and document the training course.
Drill packages – identifies the purpose, scope, and training objectives of drills.
Drill rosters – records of participation in specific drills.
Exercise manual – describes the site’s program requirements for developing and administering exercises.
Exercise packages – exercise documents under development or exercise records from exercises performed.
Exercise After Action Reports – reports to management describing the exercise scenario and results.
Exercise schedules – a multi-year plan demonstrating how the site manages exercises to ensure evaluation of all emergency management program elements on a periodic basis.
Exercise findings and corrective action plans – weaknesses identified during exercise performance and associated corrective actions.

Data Collection Activities

Document Review

Training Program Administration

A. The inspector should evaluate the training program described in the emergency plan and implementing procedures and determine whether it specifies:

Training requirements for key emergency management positions and response teams.
Examination requirements.
Record-keeping requirements to verify that training requirements are met.
Description of the training available and required for visitors, vendors, and subcontractors.
Offsite training support.
Instructor training and qualifications.
A drill program description, including evaluation and corrective actions.

B. The inspector should evaluate if training materials are maintained current by evaluating mechanisms used to update lesson plans and other instructional materials and review a sample training materials to ensure they are in use and effective. Training materials should be updated as necessary prior to its use or as required based on emergency management program changes.

C. The inspector should determine whether the training goals, organizational responsibilities, resources, and planned activities are clearly defined.

D. The inspector should review the ERAP to determine if the site is effectively implementing the training, drill and exercise program based on the ability to implement applicable schedules and manage related findings and corrective actions.

E. The inspector should evaluate the annual refresher training by determining if it addresses notification procedures, lessons learned, significant program changes, and periodic training of seldom-performed tasks.

F. The inspector should evaluate the adequacy of ERO training by determining whether it provides training to ERO member on the site’s emergency response concept of operation and instruction on position-specific tasks.

G. The inspector should determine the effectiveness in mechanisms for assigning trained individuals to the ERO duty roster by reviewing the training records of both the primary and alternate persons assigned to each key position. Use 100 percent inspections for smaller EROs or sampling techniques for larger EROs to record the level of training accomplished by non-key positions, including alternates.

H. The inspector should determine the adequacy of mechanisms used to keep an adequate number of ERO cadre members up-to-date in training by determining if the processes used to track ERO training requirements, notify members of impending or existing training deficiencies, and schedule needed training are effective.

I. The inspector should evaluate the adequacy of general employee training by determining whether it establishes training requirements that covers protective actions upon initial employment, when expected actions change, or when the emergency plan changes.

J. The inspector should evaluate the mechanisms to ensure training is provided for significant change to plans and procedures by determining whether a process links changes and revisions to a training needs analysis.

K. The inspector should evaluate the adequacy of the process for developing training materials by determining if it implements a systematic approach to training for the identification of training needs and development of the appropriate type of training. The types of training should incorporate classroom instruction, computer-based, self-study (required reading), team training, and hands on training.

Instructor Qualifications

L. The inspector should evaluate the quality of instructors by review the emergency plan or training and drills administrative document to identify the qualifications of instructors. This evaluation should also include a review of instructor training and qualification records. Important attributes to look for are:

If the instructors complete all basic instruction training courses.
If the instructors are subject matter experts in their line of instruction.
If instructors have been properly “grandfathered,” or received a waiver due to experience in accordance with program requirements.

M. The inspector should coordinate with the inspector assigned to the Offsite Response Interface and/or EPI functional areas to effectively and efficiently determine if the training available to offsite personnel is complete and accurate. See Sections 2C and 2Efor additional guidance.

N. The inspector should evaluate the training program feedback and improvement mechanisms by reviewing student feedback or course critiques and a sample of actions taken to improve training based on the critiques.

Drills

O. The inspector should determine if the drill program is comprehensive by reviewing drill packages to determine if they include drills on:

Building evacuation and/or sheltering in place.
Personnel accountability.
Protective action decision-making.
Functional drills to test training of a specific function (notification, medical response, mutual aid).
Integrated drills to test two or more functions/organizations such as the consequence assessment team with the emergency director or unified command.

P. The inspector should evaluate the quality of the drills by reviewing a sample of drill packages to determine if they:

Contain a purpose, scope, and objectives to be demonstrated.
Contain performance criteria consistent with the site emergency plan and procedures.
Are consistent with the hazards surveys and EPHAs.

Q. The inspector should determine if the drill program is used to promote program improvement by reviewing participants’ critique sheets and after-action/lessons-learned reports from drills and the mechanisms used to implement program changes. The inspector should consult the inspector assigned to the Readiness Assurance area regarding the effectiveness of a sitewide issues management process, if used for weakness identified through drills.

Exercises

R. The inspector should evaluate the adequacy of the exercise program by reviewing the emergency plan and implementing procure (or exercise manual) to verify that the site has provisions for annual facility level exercises, annual site level exercises and triennial full participation exercises.

S. The inspector should determine the quality of the exercise program by reviewing the exercise manual to determine if it prescribes provisions for the following:

Requires exercises packages to fully document exercises by including specific exercise objectives, scope, scenario narratives, participants, simulations, time lines, injects, technical data, safety and security provisions, controller and evaluator instructions, and evaluation criteria.
Provides definitions and disposition requirements for the various observation types used in the exercise program – strengths, weaknesses, findings, recommendations, observations, etc.
Describes the process used for determining whether an objective is met/not met when an objective with multiple criteria is observed with mixed rating results for the criteria.
Specifies requirements for conduct of controllers, evaluators, observers, hot washes, and critiques.
Specifies how to place administrative holds or terminate exercises.
Places limits on performing duties as a controller and evaluator.
Requires exercise training for all participants.
Requires personnel assigned to oversee safety during an exercise and specifies safety training to all participants.
Specifies exercise reporting requirements.
Specifies corrective action tracking and closure requirements.

T. The inspector should evaluate the quality of the exercise packages by reviewing recent exercise packages to determine whether:

Packages adhere to the site’s exercise manual requirements, particularly where there are findings and weaknesses requiring corrective actions.
They include objectives and criteria for each organization within the scope of the exercise.
Objectives and criteria are measurable.
They contain examples where new plans and procedures or recently established program improvements were validated.
Scenario material is consistent with the set of exercise objectives and supports a demonstration of each objective.
Simulations pertaining to participants and exercise activities are clearly identified.
Scenario material reflects facility-specific hazards, correlates technically with the facility EPHA, and is technically accurate in terms of operations and radiological, chemical, and meteorological data.
They contain provisions for safety, security, and public/media interface that are clearly identified.
They contain attendance records of all required training and briefings.

U. The inspector should evaluate the effectiveness of the exercise program in validating recent emergency management program changes and in identifying areas in need of improvement by reviewing After Action Reports to determine whether:

The exercise reports accurately reflect the conditions recorded in the completed exercise package.
Findings are correlated to applicable objectives.
Recommendations for corrective actions are provided.

V. The inspector should evaluate if the exercise program is effectively used to test all the program elements over a multiyear program by performing the following reviews.

Review the site’s master exercise schedule and assess whether it evaluates all the emergency management program elements over a 5-year period.
Determine if building evacuation exercises are conducted to ensure all workers are able to safely evacuate their work area.
Determine whether the site rotates appropriately among site facilities in their exercises.
Determine whether the site has conducted exercises as scheduled.
Determine whether each facility with the potential for a classifiable operational emergency is subjected to an annual exercise to evaluate its emergency response capability.
Determine whether site-level response organization elements and resources participate in a minimum of one exercise annually.
Determine whether communication systems from where offsite communications are performed are tested at least annually.
Determine whether offsite supporting, and potentially affected, organizations are invited to participate in an exercise at least every three years.
Determine whether each facility with a potential for a classifiable operational emergency undergoes an external evaluation at least every three years.

W. Review exercise findings and corrective action plans and, in coordination with the Readiness Assurance evaluator(s), evaluate their effectiveness in making program improvements. See Readiness Assurance Section 2G for additional guidelines.

Training, Drill, and Exercise Program Interviews

A. The inspector should determine the current status of the training, drill, and exercise programs through interviews with key personnel involve. Key personnel include the manager responsible for the emergency management program, training mangers or coordinators, drill and exercise coordinators, records management personnel, and instructors.

Exercise Evaluation

During an exercise evaluation, while the primary focus is typically on the players’ response, an assessment can be made of important aspects of the exercise program not observable during a program review. To the extent possible, review the exercise package in the same manner as a program review. Additionally, evaluate the following aspects that are only observable during an exercise evaluation.

A. The inspector should determine the adequacy of exercise participant training by attending a sample of player, evaluator, controller, and observer training and briefings to determine whether:

Instructors are well prepared to provide the necessary training.
The scope of training provides information on the scenario; simulations; environmental, safety, and security issues; administrative holds and exercise termination methods; and communications for players, controllers, and evaluators to the appropriate target audience.
Scenarios are not divulged to players.

B. The inspector should evaluate the quality of exercise execution in its ability to safely provide safe realism in the scenario and critically test the emergency management program. During the exercise, the inspector(s) should evaluate the following aspects regarding the conduct of the exercise.

The exercise evaluator organization is sufficiently staffed to evaluate demonstration of all objectives and key decisions/actions of responders.
Exercise evaluators are familiar with evaluation criteria.
Evaluators display familiarity with respective responder organizations, functions, procedures, and anticipated responder decisions/actions.
Procedures are evaluated with respect to their use by the responders; specifically, the adequacy of procedure content for the tasks performed.
Controllers provide injects as designed or instructed by the senior controller.
Controllers do not coach, play, provide unearned information, or otherwise lead players.
Controllers enforce all safety, security, environmental, or other rules established for the exercise.
The use of controllers as evaluators is minimized and is appropriate.
Communication language and equipment used by players, controllers, or evaluators is appropriate.
The use of holds and methods of exercise termination is consistent with the training provided.
Training was adequate in that no situations occurred that should have been covered in training.
The exercise was conducted safely; there were no injuries; active safety personnel were available; holds and corrective actions were appropriate; and all participants were provided with adequate provisions and protective equipment.
The exercise maximized scenario realism within safety and reasonable budgetary restraints.

C. The inspector should evaluate the ability of the exercise program to allow all participants to provide feedback by identifying strengths and weaknesses in program elements and the conduct of the exercise. Inspectors should evaluate this by attending hot washes and critique meetings following the exercise to determine whether:

Hot washes are attended by players, controllers, and evaluators and critical evaluations took place.
The exercise critique was well attended by key personnel from hot washes, included critical evaluations, and reflective of hot wash discussions. Determine whether a recorder is documenting critique minutes for feedback and improvement and management reporting purposes.

Common Deficiencies/Potential Concerns

This section identifies areas where concerns or deficiencies have been identified in previous inspections. By reviewing this information before gathering data, inspectors can be aware of these deficiencies and concerns during interviews, walk-downs, and other data gathering activities. However, the inspector should not simply focus his/her evaluation on whether these deficiencies exist at the site being evaluated, but rather should consider all aspects of this emergency management program element (including strengths and weaknesses).

Training

Changes to plans, procedures, computer software, and/or ERO position workstations are made without notifying users or establishing training needs.
Some sites have not assigned responsibility and provided training to enable personnel to be effectively sheltered by closing doors and windows and securing ventilation systems.
Minimum training and drill requirements have not been defined for all ERO members.
Criteria for successful completion of training are not defined.
Performance evaluations of each ERO member are not performed (i.e., demonstration of competency prior to being added to ERO roster).
ERO members are on the ERO roster when training deficiencies exist.
Team training is not provided.
Training on ERO position-specific tasks is not provided or is incomplete.Lessons-learned programs are not integrated with the ERO training program.
No annual refresher training is specified.
Training records are incomplete.
Lesson plans and training materials are not formally documented and maintained through a review and approval process.
New plans and procedures are implemented before the development and implementation of training.

Drills

Drills are not used as a source of lessons learned or as part of a feedback and improvement program
ERO members receive credit for annual drill participation during a drill or real event when they did not conduct any duties that would maintain their proficiency.
ERO members are identified with training deficiencies, but mechanisms are not in place to remove them from the roster or ensure that training is completed.
Scheduled drills are not completed or are too infrequent to allow all ERO members to participate annually.

Exercises

Exercise evaluators are not familiar with their evaluation criteria.
Exercise hot washes and critiques do not foster critical assessments of program elements.
Exercise findings are not managed using an effective tracking and closure process.
Exercises are not managed to ensure that all emergency management program elements are evaluated over a multi-year period.
Facility level exercises are not performed annually.
Some sites have not ensured that effective corrective actions are implemented for exercise findings.
Exercise scenarios are developed that do not represent plausible scenarios that test emergency management program elements.
Evaluation criteria are not measurable or do not support evaluation of objectives

EMERGENCY PUBLIC INFORMATION

General Information

The Department of Energy is committed to respond to and mitigate the impact of an emergency that affects the public and the environment. This commitment effects the need to effectively communicate with the public to establish facts and avoid speculation the foremost concern for site public affairs organizations. DOE Order 151.1C requires that the department establish an emergency public information (EPI) program that includes adequate plans for all emergencies at DOE/NNSA, or contractor facilities, as well as potential offsite emergencies that may involve DOE/NNSA resources or personnel. The Order further requires the provision of accurate, candid, and timely information to site workers and the public during all emergencies to establish the facts and avoid speculation. Therefore, the priority of DOE/NNSA Headquarters and site public affairs experts is to manage the emergency information process as effectively as possible, put accurate and consistent messages into play, and provide the public and the site workers with accurate information to mitigate the affects of an emergency. To achieve this goal and to ensure the development of an effective EPI program, the integration of DOE/NNSA and site public affairs organizations is crucial, as is the integration of public information and emergency management planning.

This section provides guidance for inspectors reviewing this aspect of the emergency management program. Specific areas of significance include the provision of:

Relevant Site Documents

In addition to the site documents listed in Section 1, the inspector may choose to review following site documents before or during the performance of onsite data collection.

EPI plan and procedures or checklists – establishes the site EPI program and may include the JIC program. They describe the operations, processes, tools, roles, responsibilities, EPI cadre training requirements, facilities, equipment, and supplies used by the EPI cadre. If the EPI plan does not include the JIC component, a JIC plan, procedures, and checklists/job aids should be available.

Maps of the JIC facility and onsite media center - provides the location and layout of facility space, equipment and supplies, communication lines, and other information resources delineated in the EPI plan.

Public education materials – provides information available to the public to prepare for their response to site emergencies. Public education materials include but are not limited to:

Local/state EPI/JIC plans – provides the planning basis and interface process of local or state EPI operations, facilities, equipment, and supplies used by the JIC cadre. These plans may include a description of how offsite and onsite EPI operations and the JIC interrelate and interact and the expectations of each participant.

Data Collection Activities

Document Reviews

A. The inspector should evaluate the robustness of the EPI program by reviewing the site emergency plan, EPI plans, and implementing procedures/checklists/job aids to determine whether they:

B. The inspector should evaluate the state or preparedness of the media center/JIC through reviews of plans, procedures and agreements and a media center/JIC walk-down to determine whether:

C. The inspector should evaluate the adequacy of training materials to prepare EPI personnel, media personnel, the public, and other stakeholders by reviewing available initial and refresher training drill and exercise records. See Section 2C for additional guidance in evaluating training for offsite personnel.

D. The inspector should evaluate the level of participation with the Local Emergency Planning Commission as described in Section 2C.

Interviews

A. The inspector should evaluate the familiarity EPI personnel have with the EPI program and their roles and responsibilities by interviewing personnel in key positions. Key personnel include onsite contractor and DOE/NNSA public information personnel, the EPI trainer, the emergency preparedness coordination lead, and the DOE field office public affairs manager. It is also desirable to interview one or two representatives of an offsite organization to gain their perspective of the site’s efforts in providing information. From the interviews determine whether:

Facility Walk down

A. The inspector should evaluate the state of preparedness of EPI facilities by walkdown of the site EOC and public affairs offices to determine whether equipment and supplies are available to support the media center and/or JIC operations. Important equipment includes:

B. The evaluator should determine whether the media center provides adequate space and equipment for the EPI and media by walk downs. Important space and equipment to have includes:

C. The evaluator should determine whether the JIC provides adequate space and equipment for the EPI and media by walkdowns. The number in the JIC cadre will vary depending on the severity and duration of the emergency - estimate a cadre of between 25 and 50. Assume a minimum of 100 media plus equipment (trucks, dishes, and electrical requirements) will arrive for a significant event. Space for activities and equipment such as the following should be provided:

Common Deficiencies/Potential Concerns

Section 2F

EMERGENCY RESPONSE

General Information

Emergency response activities include the execution of the site emergency plans through implementing procedures, checklists and other operator aides that are evaluated under Section 2B, Plans and Procedures, in part, Section 2C, Offsite Response Interfaces, and Section 2E, Emergency Public Information. The Training and Drill program elements, evaluated under Section 2D, prepare emergency responders to implement their plans, procedures, and other operator aides. The use of this Emergency Response section is to allow for a demonstration of the adequacy of these program elements, in an integrated fashion, to validate program strengths or identify weakness. Evaluation of these response elements is briefly discussed in the following section and more thoroughly covered in HS-63’s Emergency Management Limited Scope Performance Test Inspectors Guide, which provides guidance on planning, conducting, and evaluating emergency response performance tests. The Emergency Management Limited Scope Performance Test Inspectors Guide should be referred to when planning and conducting performance tests.

Relevant Site Documents

In addition to the site documents listed in Section 1, the inspector may choose to review following site documents before or during the performance of onsite data collection.

EALs – site-specific criteria used to detect and recognize onsite events and assign the appropriate emergency class.
Data Outputs – provides information on the methods and units of measure for data outputs. This will include meteorological equipment, field monitoring equipment, and facility radiation and chemical alarms. This information will be used to ensure that the dataprovided during a performance test is the same as that which will be used during actual event response.
Plume Model Plots – provides field data of concentrations of hazardous materials from postulated releases; should correspond with output from consequence assessment and field monitoring operations.
Department of Transportation Emergency Response Guidebook – provides response guidance for transportation events.
Forms and lists – identifying what information will be communicated and who will be notified. In addition to being formally controlled, contact lists should be routinely updated and validated.

Data Collection Activities

Document Reviews

A. The inspector should determine if observed responses are consistent with those described in the emergency plan(s) and implementing procedures.

B. The inspector should determine if adequate records are kept by LSPT personnel by reviewing forms and checklists used during LSPTs.

C. The inspector should review plans, procedures, checklists, plume plots, and other documents where performance weaknesses were observed to determine the likely cause of weaknesses.

D. The inspector should review both the initial and follow-up notification forms to ensure that:

They are standardized and specify critical information.
The emergency manager, or designee, approves the release of notification information in accordance with procedural requirements.
The emergency manager implements a review of the notification messages for classified information or for unclassified controlled nuclear information.

E. The inspector should review plume model plots to determine if appropriate decisions were made for the given information.

Facility/Equipment Walkdowns

A. During the performance of LSPTs, inspectors should observe whether tools and indicators are available to execute emergency management tasks and functions.

B. Inspectors should evaluate the equipment available on scene to support decision-making by walkdown of the command vehicle to ensure a complete up-to-date set of resources is available to perform on-scene command functions.

C. To validate observations or obtain clarifications following LSPTs, the inspector should perform walkdowns at the emergency response center.

D. At the conclusion of an LSPT scenario (or during a performance-based “discussion”), interview site emergency directors and initial decision-makers to evaluate their understanding of the ERO and its functions. Determine whether they have a clear understanding of their response roles, responsibilities, and authority.

E. At the conclusion of an LSPT scenario (or during a performance-based “discussion”), interview incident commanders to evaluate their understanding of the ERO and its functions and responsibilities. In particular, evaluate their understanding of decision-making under unified command in varying circumstances, including security events.

F. At the conclusion of an LSPT scenario (or during a performance-based “discussion”), interview onsite personnel responsible for notifications and communications to determine whether their understanding of the individual(s) responsible for notifications and communications is consistent with plans and procedures.

G. Interview building managers that are responsible for the local emergency response to ensure they have an adequate understanding of the local emergency response and their interface with the ERO.

Common Deficiencies/Potential Concerns

This section identifies areas where concerns or deficiencies have been identified in previous inspections. By reviewing this information before gathering data, inspectors can be aware of these deficiencies and concerns during performance tests, interviews, walkdowns, and other data gathering activities. However, the inspector should not simply focus his/her evaluation on whether these deficiencies exist at the site being evaluated, but rather should consider all aspects of this emergency management program element (including strengths and weaknesses).

Emergency Response Organization

Delays and/or confusion noted in responding to and mitigating an event due to weaknesses in establishing the chain of command.
ERO decisions-makers cannot apply the most applicable EAL; oftentimes they apply protective actions to an unmanageable and inappropriate population.
ERO members do not understand consequence assessment plume plots and release them to offsite authorities with errors or are not concerned with the significant consequences they might induce.
Not all offsite authorities are notified as required, and sometimes they are not provided a complete or the same set of information.
Equipment and systems in place to support site/facility personnel accountability procedures are inadequate to ensure timely and accurate determination of personnel accountability during an emergency.
EOC and other command centers have different versions of the EALs.
Classification for multiple events is not considered.
Points-of-contact lists are outdated and incomplete.
Notification forms are not correctly filled out, not corrected appropriately, or different versions are used.
Notification forms are not formalized or coordinated with offsite officials.
Notifications are not timely due to overloading of personnel with too many administration functions.
Required subsequent notifications were not made within 15 minutes of change in event status.
Insufficient information is provided during briefings of skilled emergency support personnel.
The system for rapid notification and recall does not provide for authentication and feedback.
The established communication chain for reporting and notification within the facility, site-wide, and to offsite organizations is not properly followed.
There is a breakdown in the system for tracking personnel and providing personnel accountability.
Communications equipment breaks down and backup systems are not reliable or in place.
A breakdown in communication occurs between the emergency manager and on-scene command.
Emergency information is not effectively or continuously tracked to ensure uniform understanding and provide a record of the sequence of events and actions.
Logs are not sufficiently maintained to support post-event analysis.
Emergency response managers rely on memory instead of procedures for performing tasks like categorization/classification, formulation of PA, chemical hazards, and event termination.
Emergency response tasks cannot be performed by off-normal shift personnel.
Rescue and recovery risks are not adequately assessed.
The emergency is downgraded after categorization/classification.
DOE guidelines for emergency exposure situations are not followed.

Consequence Assessment

Consequence assessments are delayed and/or not accurate.
Differences among various consequence projections are not reconciled.
Default consequence modeling parameters have not been modified to reflect site-specific requirements.
Personnel do not refine consequence projections based on updated/confirmed source term information or determine the consequences of “what if” conditions.
Consequence assessors do not have ready access to information or tools needed to perform timely and accurate assessments. Tools and information include: source term information and conversion factors; field monitoring data from various field teams; tools to overlay consequence projections onto maps showing impacted areas and receptors.
Initial decision-makers cannot interpret protective-action (PA) formulation tools.
Consequence assessors do not apply the correct PA criteria when formulating PA recommendations for decision-makers.

Protective Actions

Co-located workers are not promptly notified of PAs.
PAs are not well coordinated between the EOC and other command center locations.
Security force dispatch and response personnel lack familiarity with the emergency management concepts of isolation zones, PA zones and EPZs to ensure their safety.
Security forces do not possess personal protective equipment or are not trained in its use.

Emergency Public Information

Initial news releases are not issued within an hour following an event.
News releases contain inaccuracies and are not timely.
Technical briefers and spokespersons do not use plain English during briefings.
EPI cadre does not understand or follow information approval processes and rumor control.
EPI cadre does not understand onsite hazards and their potential effect on public health and safety.
Reluctance to activate the JIC.
The JIC organization lacks offsite coordination or input.
A DOE/NNSA official does not attend press briefings.
EPI personnel lack ready access to information and tools needed.
Inadequate workspace and resources for media at the JIC.
Recovery is not coordinated as required with offsite agencies.

Section 2G

READINESS ASSURANCE

General Information

In accordance with DOE Order 151.1, every DOE/NNSA site or facility must establish a readiness assurance program to ensure that the stated emergency capabilities are sufficient to execute the emergency plan. The readiness assurance program encompasses the site or facility, the cognizant field element, and responsible Headquarters line management. It includes the policies, plans, procedures, and processes by which the site assures its abilities to execute the emergency plan and DOE/NNSA implements its responsibilities for oversight of the site or facility program. This section provides guidance for inspectors reviewing this aspect of the emergency management program. In particular, this section provides guidance for the review of:

Appraisal and assessment programs
Evaluations
ERAPs
DOE/NNSA oversight.

While this section discusses the role of DOE/NNSA personnel in providing line management oversight of the emergency management program, it does not address the integration of DOE/NNSA personnel into the ERO or the development and conduct of exercises, which are addressed in other sections.

Relevant Site Documents

In addition to the site documents listed in Section 1, the inspector may choose to review following site documents before or during the performance of onsite data collection.

Evaluation and assessment reports – These reports contain details of both the internal (independent or self-administered) and external evaluations and assessments that have been performed. The reports of interest include those pertaining not only to the site but also to the DOE/NNSA site office.
Corrective action plans – These plans delineate actions designed to address findings (and perhaps observations) from exercises and internal or external evaluations and assessments. Of particular interest is follow-up for the corrective actions from previous Independent Oversight inspections.
Action item tracking reports – These database reports detail the status of corrective actions, including whether the action is open or closed, expected completion dates, and verification or validation activities. It is useful to obtain the report for a period of 18 to 24 months in order to observe trends in the processes for completing and closing the action items.

Data Collection Activities

Document Review

A. The inspector should determine the adequacy of the readiness assurance area through reviews of both the site contractor’s and DOE/NNSA site office’s emergency plans and implementing procedures governing assessments. Important program attributes to determine are established procedures and process for:

Scheduling and integrating external and internal assessment and DOE/NNSA self-assessment activities.
Establishing objective assessment criteria.
Performing assessments.
Assessing the severity of identified weaknesses.
Preparing and reviewing assessment reports.
Following up on findings and observations.

B. The inspector should determine the site’s ability to perform planned assessments by reviewing previous and current schedules for planned site assessments and comparing them to the available reports of completed assessments.

C. The inspector should evaluate the quality and effectiveness of completed assessments by reviewing assessment reports to determine:

The use of objective assessment criteria and objective evidence of performance.
Completion of the activities specified in the assessment schedule.
Findings and observations.
Follow-up activities.

D. The inspector should evaluate the adequacy of the DOE/NNSA oversight program through review of the DOE/NNSA site office emergency plan and implementing procedures governing oversight activities.

E. The inspector should evaluate the quality of processes used to review and approved key emergency management documents by reviewing procedures governing the approval of emergency plans, EPZs and EPHAs. These documents should include:

Roles and responsibilities of DOE/NNSA personnel
A description characterizing the thoroughness of reviews.

F. The inspector should evaluate the effectiveness of DOE/NNSA approval reviews based on reviews of a sample of approved exercise packages, after action reports, final emergency reports, emergency plans, ERAPs, EPZs, EPHAs, and hazard surveys.

G. The inspector should verify that the site has adopted DOE’s Oversight policy by reviewing the site operating contract to ensure inclusion of DOE Order 226.1.

H. The inspector should evaluate the process for developing and approving ERAPs by reviewing both the site contractor’s and the DOE/NNSA site office’s processes and procedures for preparing the ERAP and determining process effectiveness by reviewing approved ERAPs. Positive attributes include:

Roles and responsibilities of site and DOE/NNSA personnel for the preparation, review, approval, and submittal of the ERAP
The ERAP contents are complete and accurate.

I. The inspector should evaluate the effectiveness of the use of contract performance measures by reviewing the applicable DOE/NNSA site office processes and procedures. Specifically:

Examine the roles and responsibilities of DOE/NNSA personnel for preparing, approving, and tracking contract performance measures.
- Review the interactions in establishing program performance measures and milestones
- Review the processes and procedures used to track and evaluate performance.
Review suitability of performance measure deliverables and due dates.
Review documentation of the outcome and evaluation of activities related to the performance measures.

J. The inspector should evaluate the effectiveness of the site’s corrective action process by reviewing both the site and DOE/NNSA site office processes and procedures governing corrective actions. Specifically:

Verify that procedures establish clear roles and responsibilities for:
- Use and conduct of root cause analysis
- Determining corrective actions to address findings and/or observations
- Establishing overall management responsibility, as well as responsibility for implementation of the corrective action
- Assigning a schedule for corrective actions, including start and finish dates
- Verifying the effectiveness of corrective actions.
Examine the processes and procedures for:
- Establishing roles and responsibilities for issues management, including overall management responsibility
- Periodically reviewing and reporting corrective action status, including follow-up mechanisms for overdue actions
- Managing changes to the schedule and action items
- Verifying and validating completed actions.
Review reports of the status of corrective actions.
Examine the site performance in overall corrective action implementation, including:
- Meeting established schedules
- Revising schedules, when appropriate
- Executing the corrective action plan
- Establishing reasonable evidence of closure
- Completing verification and validation activities.

K. The inspector should evaluate the training and procedures used to guide DOE/NNSA responders during an emergency by reviewing the applicable training, procedures, and checklists provided to the DOE/NNSA site office personnel.

L. The inspector should evaluate the effectiveness of the lessons learned program in promoting improvements by reviewing the site lessons learned program. Specifically:

Verify that site participates in DOE/NNSA Corporate Lessons Learned Program.
Confirm that lessons learned from training, drills, and actual responses are incorporated and tracked in the site lessons learned program.

Interviews

A. The inspector should obtain a status of the self-assessment program and the emergency management (corrective action) coordinator’s understanding of the program by interviewing the site contractor’s emergency management (corrective action) coordinator. Specifically:

Discuss the implementation of the site self-assessment program, including:
- Roles and responsibilities
- Overall strategy and approach to meeting the requirements
- Scheduling issues, including integration with other assessments and inclusion of all the emergency management functional areas
- Performance issues, including the choice and use of objective criteria and training of assessment personnel
- Mechanism used to provided emergency management program status information and issues to senior management
- Interactions with DOE/NNSA site office personnel in conjunction with assessment activities.
Discuss the implementation of the site lessons learned program. Include such topics as:
- Roles and responsibilities
- Identification and selection of lessons learned.
Discuss the processes and procedures utilized for preparing, reviewing, and approving the ERAP, including:
- Roles and responsibilities of site personnel
- Overall process and approach used to prepare the ERAP
- Perception of usefulness
- Interactions with DOE/NNSA site office personnel during preparation and approval.

B. The inspector should obtain a status of the site oversight, contractor assessment and self-assessment program, and the DOE/NNSA site office emergency management coordinator’s understanding of the programs by interviewing the DOE/NNSA site office emergency management (corrective action) coordinator. Specifically:

Discuss the implementation of the site oversight, contractor assessment and self-assessment program, including:
- Roles and responsibilities
- Overall strategy and approach to meeting the requirements
- Scheduling issues, including integration with other assessments and inclusion of all the emergency management functional areas
- Performance issues, such as the choice and use of objective criteria and the training of assessment personnel
- Documentation of oversight activities
- Mechanism used to communicate emergency management program status information and issues to senior management at the site office
- Method used to provide formal, routine feedback to the contractor on their emergency management performance
- Provision of direction to contractor on implementing emergency management policy and requirements
- Status of implementation of DOE Order 151.1 and 226.1
- Method used to track timely updates of hazards surveys and EPHAs
- Interactions with DOE/NNSA Headquarters office personnel regarding the statues of the site emergency management program.
Discuss the processes and procedures utilized for preparing, reviewing, and approving the ERAP, including:
- Roles and responsibilities of DOE/NNSA personnel
- Overall process and approach used to prepare the ERAP
- Perception of usefulness.
Discuss the processes and procedures used for preparing, approving, and tracking contract performance measures, including:
- Roles and responsibilities of DOE/NNSA personnel for the preparation, approval and tracking of performance measures
- Interactions used to establish program performance measures and milestones
- The effectiveness of the performance measures in achieving the desired goals.
Discuss the processes and procedures utilized for preparing, reviewing, and approving the emergency planning documents, such as the hazards summaries and EPHA(s). Topics may include:
- Roles and responsibilities of both site personnel and DOE/NNSA personnel
- Availability and utilization of qualified personnel to perform the required reviews
- Interactions with DOE/NNSA Headquarters and other support personnel in the review and approval process.

C. The inspector should evaluate the DOE/NNSA Headquarter’s office perspective of the site oversight and assessment program by interviewing the DOE/NNSA Headquarter’s office personnel responsible for the site. Specifically:

Discuss the implementation of the site oversight and assessment program, including:
- Roles and responsibilities
- Overall strategy and approach to meeting the requirements
- Involvement in oversight, assessment, and exercise evaluation activities
- Processes and procedures for receiving, reviewing, and incorporating ERAP inputs into the Headquarters reports.

Common Deficiencies/Potential Concerns

At some sites and offices, personnel responsible for the management and oversight of the emergency management program do not have experience or training commensurate with the breadth and depth of the program. In addition, personnel may not obtain adequate support services. These weaknesses are reflected in the planning and response documents, particularly the technical adequacy of the hazards surveys and EPHA. Some specific proficiency problems that may be observed include:

Technical document reviews are not performed in a timely or rigorous manner, resulting in planning document weaknesses.
Personnel, whose training and experience provide insufficient knowledge of the standards and criteria for satisfactory performance, conduct unchallenging site and DOE/NNSA emergency assessments.

At several sites/facilities and DOE/NNSA site offices, oversight and assessment activities were not being performed as required by site or DOE/NNSA policies and procedures. Weaknesses observed include:

Insufficient numbers of assessments to cover the required functional areas scheduled over the allotted time period.
Scheduled assessments not performed.
Corrective actions for identified deficiencies not implemented or not implemented in a timely fashion.
Inattention to the self-assessment program.
Lack of integration of the assessment and oversight programs to achieve the desired degree of feedback.
Insufficient numbers of DOE/NNSA oversight activities scheduled and/or performed.
Insufficient management attention to feedback and improvement activities.

Section 3

ANALYZING DATA AND INTERPRETING RESULTS

General Information

This section provides guidelines to help inspectors analyze data and interpret the results of data collection. The guidelines include information on the analysis process, including factors to consider while conducting an analysis. Information is also included on the significance of potential deficiencies, as well as suggestions for additional activities when deficiencies are identified. After completing each activity, inspectors can refer to this section for assistance in analyzing data and interpreting results and for determining whether additional activities are needed to gather the information necessary to accurately evaluate the system. When analyzing the data collected on a particular aspect of the site emergency management program, it is important to consider both the individual elements of the emergency management program and the program as a whole. In other words, the presence of significant weaknesses in a single area of an emergency management program element does not necessarily mean that the entire emergency management program is ineffective. However, a number of relatively insignificant systemic deficiencies can collectively degrade the overall effectiveness of the emergency management program. This is why integration among program element inspection teams is so important. It provides a look at the “big picture” within the framework of the site mission when determining whether the overall emergency management program is effective.

Analyzing Data

Data review consists of sorting out and logically grouping all validated data collected for each program element during each phase of the inspection (remembering that data is collected during the planning process as well as the conduct phase). Although the inspection team is generally aware of most of the data, not all team members will be familiar with all data collected. Therefore, it is important for the inspection team to review data at the end of each day to begin to develop a comprehensive picture of how effectively the emergency management program meets requirements. This can be best accomplished while preparing for the nightly inspection team meeting. In this way individual inspectors of the emergency management program can come together to discuss each validated data point, begin the process of analysis, and identify the impact as it may exist at that point in time (recognizing that additional data may eliminate, mitigate, or increase the impact of a particular concern). Generally, it is helpful to arrange the data according to positive or negative features. This will aid in clearly identifying strengths, weaknesses, and positive or negative trends. Proper organization and thorough review of all inspection data are essential to analysis and report preparation.

Potential Impacts on Other Program Elements

Many of the program elements are interrelated. It is helpful for the inspector to properly identify the dominant cause(s) of a program’s weakness for the purpose of writing a finding statement in a manner that leads the site to a corrective action plan that addresses the systemic cause(s) rather than the symptoms. Examples of interrelationship among program elements are:

Hazards Surveys and EPHAs. The extent and impacts of hazardous material releases define the need for plans, procedures, equipment, offsite resources, and staff to effectively protect personnel and the environment and mitigate releases. Weaknesses in the hazards surveys and EPHAs can affect all any of the program elements to plan, prepare, and respond to a hazardous material release.

Plans and Procedures. Response issues related to plans, procedures, and other response documents are typically found to have factual or editorial errors that degrade the effectiveness of response because they are not internally consistent, or are not current. Additionally, some response issues may be indicative of inadequate cooperative/joint planning between different organizations on how to respond to certain circumstances.

Offsite response interfaces. The response by offsite organizations is contingent on the type and severity of the emergency as agreed upon through MOU. MAA, and LOU. These agreements inform each party how to prepare one another, notification and asset activation protocols, and how respond to the different types of plausible emergencies. Site actions within these agreements should be stipulated in the emergency plan and implementing procedures. Preparedness of offsite responses are a part of the training and exercise emergency management program element.

Training, Drills, and Exercises. Responders’ unfamiliarity or inability to respond effectively is indicative of weakness that may originate in the training, drill, and exercise programs. This may be the result of incorrect or no training, use of improper training settings, training that is too infrequent, or absence of non-critical performance evaluation techniques.

Emergency Response. Weaknesses noted during the limited scope performance tests generally can be tied to either problems in plans and procedures or in training and drills.

Readiness Assurance. Weaknesses in the Readiness Assurance area may impact any of the emergency management program elements due to the inability of identifying weakness or effectively correcting them.

Interpreting Results

The process for analyzing results begins with the first document to be reviewed, briefing received, or person interviewed during planning. It is not completed until the final inspection report is disseminated. By recognizing this concept early in the inspection process, the inspection team can enhance the completeness and usefulness of its analysis. The information collected for each of the emergency management program elements is reviewed to determine whether the overall emergency management program complies with the requirements in DOE orders.

In addition to mere compliance, the analysis process involves the team members’ critical consideration of all inspection results, particularly identified strengths and weaknesses or deficiencies, framed within the parameters of the site mission. Analysis should lead to a logical, supportable conclusion regarding how well the emergency management program is meeting the required standards and satisfying the intent of DOE/NNSA requirements. A workable approach is to first analyze each program element individually. The program element inspection tools (Sections 2A through 2H) provide guidance to assist in this evaluation. The results can then be integrated to determine the effects of the program elements on each other and, finally, the overall status of the program.

As mentioned before, it is important to weigh the significance of a weakness or deficiency in light of the entire system. If there are no deficiencies, or those identified do not impact the rating, the analysis is relatively simple. In this event, the analysis is a summary of the salient inspection results supporting the conclusion that emergency management program needs are being met. If compensatory systems or measures were considered in arriving at the conclusion, they should be discussed in sufficient detail to clearly establish why they counterbalance the identified deficiencies. If there are negative findings, weaknesses, deficiencies, or standards that are not fully met, the analysis must consider the significance and impact of these factors. The deficiencies must be analyzed both individually and collectively, then balanced against any strengths or mitigating factors to determine their overall impact on the site emergency management program’s ability to meet DOE/NNSA requirements and site mission objectives. Other considerations include:

Whether the deficiency is isolated or systemic
Whether the operations office or contractor management previously knew of the deficiency and, if so, what action was taken
The importance or significance of the standard affected by the deficiency
Mitigating factors, such as the effectiveness of other emergency management program elements, that could compensate for the deficiency
The deficiency’s actual or potential effect on the ability of the site to protect workers and the public in the event of an emergency.

Ratings

HS-63 assigns ratings to the supporting elements of a site’s emergency management program. The conclusions reached through analysis of inspection results lead to the assignment of ratings. Independent Oversight uses three rating categories: Effective Performance, Needs Improvement, and Significant Weakness, which are also depicted by the colors green, yellow, and red, respectively. The HS-63 rating system is described in more detail in the Emergency Management Oversight Appraisal Process Guide.

Findings

Inspection findings are the primary means of identifying those elements of the emergency management program that are having a significant negative impact on the effectiveness of the overall program. The inspection team is expected to exercise judgment in determining findings, omitting minor and non-systemic items, and limiting formal findings to items of significance. Further guidance for formulating findings is located in the Emergency Management Oversight Appraisal Process Guide.

APPENDIX A REFERENCES

American Industrial Hygiene Association, Emergency Response Planning Guidelines and Workplace Environmental Exposure Level Guidelines Handbook

DOE Order 226.1, Implementation of Department Of Energy Oversight Policy (September 2007)

HS-63 Emergency Management Limited Scope Performance Test Inspectors Guide (February 2008)

Office of Independent Oversight and Performance Assurance Appraisal Process Protocols (January 2002)

Office of Independent Oversight Emergency Management Oversight Appraisal Process Guide (January 2007)

Secretary of Energy Directive, Notification and Reporting Procedures for Emergency and Other Significant Events (August 27, 1997)

AEGL	Acute Exposure Guideline Level
CAP	Corrective Action Plan
DOE	U.S. Department of Energy
DSA	Documented Safety Analysis
EAL	Emergency Action Level
EMG	Emergency Management Guide
EOC	Emergency Operations Center
EPA	Environmental Protection Agency
EPHA	Emergency Planning Hazard Assessment
EPI	Emergency Public Information
EPZ	Emergency Planning Zone
ERAP	Emergency Readiness Assurance Plan
ERO	Emergency Response Organization
ERPG	Emergency Response Planning Guidelines
FHA	Facility Hazards Analysis
HS-60	Office of Independent Oversight
HS-63	Office of Emergency Management Oversight
JIC	Joint Information Center
LEPC	Local Emergency Planning Committee
MOA	Memoranda of Agreement
MOU	Memoranda of Understanding
NARAC	National Atmospheric Release Advisory Center
NIMS	National Incident management System
NNSA	National Nuclear Security Administration
NRP	National Response Plan
OSHA	Occupational Safety and Health Administration
PRA	Probabilistic Risk Assessment
SAR	Safety Analysis Report
SERC	State Emergency Response Committee
TSR	Technical Safety Analysis
TEEL	Temporary Emergency Exposure Limit