Detailed Information on the
Federal Emergency Management Agency: Readiness Assessment

Review and update the National Response Plan and National Incident Management System with Federal, State and local partners.

Develop comprehensive readiness and training plans once FEMA and the DHS Preparedness Directorate have completed the 2007 restructuring.

Improve EMI's Curriculum Management System to better align curriculum and training with FEMA mission, target audience, national policies (NIMS, NRF, TCL, NPG, etc.)

Baseline the new efficiency measure for "cost of student training hour" and develop a cost effectiveness model.

Measure: Percent of Federal, State, Local and Tribal Governments compliant with the National Incident Management System (NIMS)

Explanation:Federal Agencies: Federal Agencies are required to identify a point of contact (POC) within their agency to act as a liaison with the NIMS Integration Center (NIC). In addition, they were also required to create a NIMS Implementation Plan, incorporate NIMS into their respective emergency operations plans and train all appropriate personnel in the NIMS standard training curriculum. To date, 100% of Federal agencies have identified a POC, submitted an implementation plan and trained some of their personnel. Since the emergency operations plans have not been evaluated, it is not clear whether all agencies have incorporated NIMS into their existing emergency operations plans. The NIMS Integration Center met with each agency to provide technical assistance toward compliance efforts. A Federal Peer review compliance program also was established in June 2006, which is expected to increase the effectiveness of compliance measurements. State and Local Governments: NIMS implementation and compliance at the State and local levels is a condition of receiving any federal preparedness assistance. States have 23 compliance requirements and local governments have 17 compliance requirements. Local governments report their compliance efforts to their respective State and the State NIMS Compliance for FY2005 and FY 2006 was determined through a "self-certification" process. By definition, all states submitting certification forms are considered to be NIMS compliant for FY 2005 and FY 2006. As of 1 November, 2005, all 56 States and territories had certified they were in compliance for the FY 2005 fiscal year. Local governments are not required to be in compliance until September 30, 2006.The self-certification process was chosen for the first two years of the program to allow State and local governments to receive adequate technical assistance and support to meet NIMS compliance requirements. Full NIMS implementation is a dynamic and multi-year phase-in process with important linkages to the National Response Plan (NRP) and HSPD-8, National Preparedness. Beginning in FY 2007, compliance will be performanced-based and compliance will be monitored by the DHS Office of Grants and Training (OGT). This measure was chosen because by satisfying NIMS compliance requirements, States, tribal governments, territories and local governments are acknowledging their on-going responsibilities and support to ensure the future full NIMS implementation. While compliance in FY 2005 and 2006 was through a self-certification system, compliance with a performance-based metrics compliance system, which will come on line in 2007, can be tracked and reported accurately.

Measure: Cost per student hour of training.

Explanation:EMI developed a cost efficiency measure with FY 2006 as the baseline year in order to monitor costs while achieving its long term performance goal. This is similar to those used by other DHS training organizations such as FLETC and CDP; however, they use weekly and per student costs. EMI chose the hourly measure in order to accommodate the independent study training which is included along with resident training in this measure.

Measure: 80% of respondents reporting they are better prepared to respond to disasters and emergencies as a result of EMI training.

Explanation:EMI sends a follow-on survey to students 90 days after completing the training to determine if they are better prepared to respond to disasters and emergencies as a result of EMI training. It is a recognized fact that it is difficult to measure the effectiveness of training due to other factors which impact the outcome. EMI students may not have an opportunity to test their newly acquired knowledge and skills if they do not experience a disaster or emergency in their community following the training.

Is the program purpose clear?

Explanation: Statute and Policy documents establish the Readiness program to ensure that all levels of government across the Nation have the capability to work efficiently and effectively together during an incident, using a national approach to domestic incident management. Readiness accomplishes this goal through two major outcome-oriented sub-programs-- the National Incident Management System (NIMS) and the Emergency Management Institute (EMI). Homeland Security Presidential Directive (HSPD)-5 and DHS Management Directive 9500 established the NIMS Integration Center under the Under Secretary of Emergency Preparedness and Response, now the Under Secretary of FEMA, to provide strategic direction and oversight of the NIMS, supporting maintenance and the continuous refinement of the system and its components over the long term. EMI serves as the national focal point to develop and deliver of emergency management training to enhance the response capabilities of Federal, State, local and tribal government officials, volunteer organizations, and the public and private sectors and to minimize the impact of disasters on the American public.

Evidence: HSPD-5; National Incident Management System Document; Department of Homeland Security Management Directive MD-9500; Title VI of the Robert T. Stafford Disaster Relief; Emergency Assistance Act, as amended by Public Law 106-390, October 30, 2000.

Does the program address a specific and existing problem, interest, or need?

Explanation: To accomplish DHS Strategic Goal objective 3.7 "Strengthen nationwide preparedness and mitigation against acts of terrorism, natural disasters, or other emergencies" the Readiness program through the NIMS Integration Center and the Emergency Management Institute develops training, standards & guidelines needed to enable and enhance management capabilities of Federal, State, Local, and tribal governments, private sector, and nongovernmental organizations to work together effectively and efficiently to prepare for, prevent, respond to, and recover from domestic incidents. Implementation of NIMS and effective emergency management training will increase coordination at all levels of government in the event of a domestic incident.

Evidence: HSPD-5; National Incident Management System Document; Department of Homeland Security Management Directive MD-9500; White House Katrina After Action Report, DHS IG Katrina Report; NETC admissions system; independent study database.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Department of Homeland Security has identified the NIC to be solely responsible for the oversight and development of the National Incident Management System. The NIMS Integration Center (NIC) is a coordinating organizations that works directly with other Federal Departments and Agencies, State and local governments, and private sector organizations to ensure there are no redundant or duplicate efforts in the implementation of the National Incident Management System and the National Response Plan. As FEMA's training arm, EMI trains FEMA's federal disaster workforce, federal agencies, and State, local, and tribal emergency management officials. EMI conducts extensive coordination of its training program at the Federal, State, local, and tribal levels, is an active member of the DHS Training Leaders Council, and routinely works with DHS components such as the Office of Grants and Training, U.S. Fire Administration, and Customs and Border Protection, as well as other Federal agencies such as Centers for Disease Control, Veterans Affairs, Department of Defense, and Department of Health and Human Services.

Evidence: HSPD-5; National Incident Management System Document; Department of Homeland Security Management Directive MD-9500; Active participant on DHS Training Leaders Council, EMI Course Catalog, and Title VI of the Robert T. Stafford Disaster Relief and Emergency Assistance Act, as amended by Public Law 106-390, October 30, 2000.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program design is generally free of design flaws. There is no strong evidence that another approach or mechanism would be more efficient or effective to achieve the intended purpose. Following Hurricane Katrina's numerous After Action Reports, the National Incident Management System is being upgraded to incorporate recommendations from the reports. Readiness exercises and training courses will be revised to incorporate recent changes to the NRP. The NIMS Integration Center (NIC) as a result of internal reviews and Katrina After Action Reports will lead an interagency effort to develop and resource a deliberative, integrated and Federal Planning and execution system to meet the requirements of the revised NIMS and National Response Plan. The NIC is developing performance measures and metrics to allow objective assessment of the NRP and NIMS implementation status for all departments and agencies, and state and local governments. They are working with all federal departments and agencies to ensure NIMS compliance and NRP implementation and to ensure that the departments and agencies align their planning and training to the NIMS. Hurricane Katrina After Action reports have served to refine and improve upon the existing design of the NIMS Integration Center and Emergency Management Institute.

Evidence: Department of Homeland Security Management Directive MD-9500; Training evaluation results; National Response Plan (NRP); Notice of Changes to the NRP; Hurricane After Action Reports: White House Homeland Security Counsel's Report, Inspector General's Report; Senate Government Affairs Report; and House of Representatives Reports.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The design of the Readiness program with the NIMS Integration Center (NIC) and Emergency Management Institute (EMI) is effective in addressing the program's purpose. NIC has four branches: Standards & Resources Branch, Training & Exercise Branch, System Evaluation and Compliance Branch, and Technology & Research and Development Branch; each branch is designed to perform the functions outlined in the DHS Management Directive MD-9500. The organizations design allows for the development of a nationwide approach for Federal, State, and local governments to work effectively and efficiently together to prepare for, respond to, and recover from domestic incidents, regardless of cause, size, or complexity. EMI's training is designed for delivery in a traditional resident classroom setting, independent study on-line training, and nationwide distribution of centrally developed training materials for delivery at the State and local level to maximize EMI's ability to reach its intended audiences.

Evidence: HSPD-5; National Incident Management System Document; MD-9500; Dept. of Education Emergency Response Crisis Management & Grant Program Meetings; Credentialing Discipline Groups; Resource Typing Discipline Groups; Compliance Working Group; United Southern & Eastern Tribes (USET) Meetings; pending MOU with DHS Office of Grants & Training; Training evaluation results.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Readiness program has long term goal for both the NIMS Integration Center (NIC) and the Emergency Management Institute (EMI). The long term goal of the NIC is to have Federal, State, local and Tribal governments achieve and maintain compliance with the requirements of the National Incident Management System (NIMS) to ensure responders at all levels work together more effectively to manage domestic incidents, regardless of cause, size, or complexity. Federal Agencies are required to identify a point of contact (POC) within their agency to act as a liaison with the NIMS Integration Center (NIC). In addition, they must also create a NIMS Implementation Plan, incorporate NIMS into their respective emergency operations plans and train all appropriate personnel in the NIMS standard training curriculum. A Federal Peer review compliance program was established in June 2006, which is expected to increase the effectiveness of compliance measurements. NIMS implementation and compliance at the State and local levels is a condition of receiving any federal preparedness assistance. States have 23 compliance requirements and local governments have 17 compliance requirements. Local governments report their compliance efforts to their respective State and the State NIMS Compliance for FY2005 and FY 2006 was determined through a "self-certification" process. By definition, all states submitting certification forms are considered to be NIMS compliant for FY 2005 and FY 2006. As of 1 November, 2005, all 56 States and territories had certified they were in compliance for the FY 2005 fiscal year. Local governments are not required to be in compliance until September 30, 2006. The self-certification process was chosen for the first two years of the program to allow State and local governments time to receive adequate technical assistance and support to meet NIMS compliance requirements. Full NIMS implementation is a dynamic and multi-year phase-in process with important linkages to the National Response Plan (NRP) and HSPD-8, National Preparedness. Beginning in FY 2007, compliance will be performance-based and compliance will be monitored by the DHS Office of Grants and Training (OGT). This measure was chosen because by satisfying NIMS compliance requirements, States, tribal governments, territories and local governments are acknowledging their on-going responsibilities and support to ensure the future full NIMS implementation. While compliance in FY 2005 and 2006 was through a self-certification system, compliance with a performance-based metrics compliance system, which will come on line in 2007, can be better tracked and reported accurately. EMI's long term performance goal is to have 80% of respondents reporting they are better prepared to deal with disasters and emergencies as a result of the EMI training. In addition, EMI has developed a cost efficiency measure of $7 per student hour with FY 2006 as the baseline year in order to monitor costs while achieving the long term performance goal. While it is unrealistic to measure overall disaster readiness the method of self-certification of an individuals post training readiness demonstrates the ability of EMI to improve the nations ability to respond to disasters. EMI also has an internal annual performance goal to reach its capacity of 14,000 students who receive resident training and over 1 million students trained using EMI's independent study program in order to reach the maximum number of students to improve the nation's capability to deal with disasters and emergencies. This measure of students trained has a proportionate impact on enhancing the nation's emergency management capabilities.

Evidence: FY06-ll FYHSP Report; FY07-l2FYHSP Report; FEMA Budget Narratives and Quarterly Performance Reviews.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Readiness program has set a target of 100% of Federal, State, local and tribal government reporting compliance to NIMS standards for each fiscal year. NIMS requirements are revised, evaluated, and updated annually. Through EMI, the Readiness program has an annual and long-term goal to achieve and maintain 80% in FY 2006-11 and 85% beginning FY 2012 of respondents reporting that they are better prepared to respond to disasters and emergencies as a result of their EMI training. Currently there are no specific training benchmarks against which to measure this. In FY 2005, 23% of respondents reported that they have not experienced a disaster or emergency after attending EMI training which, therefore, does not present them with an opportunity to test their newly acquired knowledge and skills. EMI has no control over such factors. EMI is currently in the process of establishing a baseline for a cost efficiency measure to determine cost per student hour of training delivered with a target of $7.00.

Evidence: FY06-ll FYHSP Report; FY07-l2 FYHSP Report; FEMA Budget Narratives and Quarterly Performance Reviews.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The Readiness program has long term/annual goals for both the NIMS Integration Center (NIC) and the Emergency Management Institute (EMI). The long term goal of the NIC is to have Federal, State, local and Tribal governments to achieve and maintain compliance with the requirements of the National Incident Management System (NIMS) to help ensure responders at all levels work together more effectively to manage domestic incidents, regardless of cause, size, or complexity. In FY 2005, Federal Agencies were required to identify a point of contact (POC) within their agency to act as a liaison with the NIMS Integration Center (NIC). In addition, they were also required to create a NIMS Implementation Plan, incorporate NIMS into their respective emergency operations plans and train all appropriate personnel in the NIMS standard training curriculum. NIMS implementation and compliance at the State and local levels is a condition of receiving any federal preparedness assistance. States have 23 compliance requirements and local governments have 17 compliance requirements. New compliance requirements will be added as well as old ones revised each fiscal year to evaluate NIMS Compliance. Local governments report their compliance efforts to their respective State and the State NIMS Compliance for FY2005 and FY 2006 was determined through a "self-certification" process. By definition, all states submitting certification forms are considered to be NIMS compliant for FY 2005 and FY 2006. As of 1 November, 2005, all 56 States and territories had certified they were in compliance for the FY 2005 fiscal year. Local governments are not required to be in compliance until September 30, 2006.The self-certification process was chosen for the first two years of the program to allow State and local governments to receive adequate technical assistance and support to meet NIMS compliance requirements. Full NIMS implementation is a dynamic and multi-year phase-in process with important linkages to the National Response Plan (NRP) and HSPD-8, National Preparedness. Beginning in FY 2007, compliance will be performance-based and compliance on the previous years submission will be monitored by the DHS Office of Grants and Training (OGT). This measure was chosen because by satisfying NIMS compliance requirements, States, tribal governments, territories and local governments are acknowledging their on-going responsibilities and support to ensure the future full NIMS implementation. While compliance in FY 2005 and 2006 was through a self-certification system, compliance with a performance-based metrics compliance system, which will come on line in 2007, can be tracked and reported accurately. EMI's long term performance goal is to have 80% of respondents reporting they are better prepared to deal with disasters and emergencies as a result of the EMI training. In addition, EMI has developed a cost efficiency measure of $7 per student hour with FY 2006 as the baseline year in order to monitor costs while achieving the long term performance goal. While it is unrealistic to measure overall disaster readiness the method of self-certification of an individuals post training readiness demonstrates the ability of EMI to improve the nation's ability to respond to disasters. EMI also has an internal annual performance goal to reach its capacity of 14,000 students who receive resident training and over 1 million students trained using EMI's independent study program in order to reach the maximum number of students to improve the nation's capability to deal with disasters and emergencies. This measure of students trained has a proportionate impact on enhancing the nation's emergency management capabilities.

Evidence: FY06-11 FYHSP Report; FEMA Budget Narratives and Quarterly Performance Reviews; NETC Admissions system for resident training statistics and EMI Independent Study Program Database for IS completions.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: During FY 2005 100% of State governments reported they were compliant with the 2005 NIMS requirements. The NIMS Integration Center has a long-term/Annual goal to have 100 % Federal, State, and local governments reporting they are NIMS compliant. NIMS compliance will ensure better prepared response organizations that will benefit public health and safety by a strengthened national capability to prepare for and recover from an incident or disaster. In 2004 survey results for reported that 83% of respondents reported that they were better prepared as a result of their training. This goal includes a roll-up of training activity for EMI, FEMA's Learning Development Branch, and the National Fire Academy. In 2005 84.3% of respondents reported that they were better prepared as a result of their training. This goal includes a roll-up of training activity for EMI, and FEMA's Learning Development Branch. EMI's long term goal is to have 80% for FY06-11 and 85% in FY 2012 of respondents reporting that they are better prepared to deal with disasters and emergencies as a result of training. This goal includes a roll-up of training activity for EMI, and FEMA's Learning Development Branch. EMI is currently in the process of establishing a baseline for a cost efficiency measure to determine cost per student hourof training delivered with a target of $7.00. EMI's resident training program has grown from 7,000 students in FY 2000 to over 14, 000 in FY 2005. EMI's independent study (IS) program has grown from 65,000 completions in FY 2000 to over 1 million in FY 2005.

Evidence: FY06-11 FYHSP Report; FY07-12 FYHSP Report; NIMS Introductory Program Briefing; FEMA Budget Narratives and Quarterly Performance Reviews.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The Readiness program works directly with Federal, State, local and Tribal governments to ensure responders at all levels work together more effectively to manage domestic incidents, regardless of cause, size, or complexity. All Federal, State, local, and Tribal governments contribute data and input toward the annual and long-term assessments of the program goals. The NIC also works with academic institutions under cooperative agreements to work toward the annual and Long-term goals of the NIC. EMI contracts out services for training development and instructional delivery. Statements of work are written to ensure that the outcome of the contract will achieve our performance goal. Partners recruit target audience to participate in training.

Evidence: FY06-11 FYHSP Report; NIMS Introductory Program Briefing; FEMA Budget Narratives and Quarterly Performance Reviews.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The NIC conducts annual field monitoring with the Federal agencies and departments and 56 states and territories to assess the implementation of NIMS and to identify areas of NIMS non-compliance. The NIC will also use the annual monitoring to revise and develop metrics and improve upon NIMS implementation. The successful implementation of the NIMS will reduce the loss of life and property in an event. Readiness relies on external stakeholders to identify inconsistencies and errors in the NIMS compliance documents. Changes and updates to the NIMS documents will occur as a result of establishing and maintaining advisory committees and stakeholder working groups. EMI has an ongoing evaluation process built into its training development and delivery process. This includes front-end analysis to determine if training will correct the deficiency identified, focus groups including accomplished performers from the target audience, classroom testing, end-of-course evaluation, and follow-on surveys. Evaluation results are continually reviewed and incorporated as appropriate into EMI's training program. EMI does not have the financial resources to contract out an independent or third party evaluation of its programs. As part of the DHS Inspector General's review of FEMA's performance in Hurricane Katrina, EMI provided data for the past 10 years.

Evidence: FYO6-11 FYHSP Report; NIMS Introductory Program Briefing; FEMA Budget Narratives and Quarterly Performance Reviews; Course Evaluation results.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The NIC & EMI program budgets are developed through a strategic planning process at the Component and Departmental level that requires program mission to be linked to specific program objectives which are tied to program goals and objectives. The DHS Future Years Homeland Security Program (FYHSP) include program performance measures and targets, annual milestones information, and reflect alignment of programs and associated resources to the goals and objectives in the DHS Strategic Plan.

Evidence: FYO6-11 FYHSP Report; FYO7-12 FYHSP Report; DHS Planning, Programming, Budget, and Execution system; FEMA Budget Narratives and Quarterly Performance Reviews.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Readiness does not have any identified strategic planning deficiencies. The NIC regularly reviews its program for deficiencies and has a strategic planning process to correct any that are identified. Specifically the NIC annually evaluates the NIMS requirements. NIMS Compliance in FY 2005 and FY 2006 was through a self-certification system allowing State and Local government to receive adequate technical assistance and support to meet NIMS compliance requirements. Beginning in FY 2007 compliance will be measured with a performance-based performance metrics. The NIC in FY 2006 & FY 2007 will be incorporating recommendation from Hurricane Katrina After Action Reports to refine and improve upon the NIC organization. EMI does not have any identified strategic planning deficiencies. EMI regularly reviews its program for deficiencies and has a strategic planning process to correct any that are identified by the FEMA program offices. Additionally, the Remedial Action Management Program (RAMP) is designed to resolve important management issues. All FEMA program offices, including the Readiness program, are assigned issues through RAMP and must develop action plans designed to address program shortfalls. Many issues identified in After Action reports have been assigned to the RAMP process.

Evidence: FYO6-11 FYHSP Report; DHS Planning, Programming, Budget, and Execution system; FEMA Budget Narratives and Quarterly Performance Reviews.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Readiness program compiles integrated performance data quarterly, including targets progress, remedy of shortfalls and provides the basis to adjust program priorities and reallocated funds. NIC works in coordination with the NIMS Support Center and the Department of Homeland Security, Office of Grants & Training to monitor previous years submission the implementation of NIMS at the state and local level. The NIC program is currently in the process of developing monitoring/review strategies for Federal departments and agencies. EMI has an ongoing evaluation process built into its training development and delivery process to obtain feedback from participants. This includes front-end analysis to determine if training will correct the deficiency identified, focus groups including accomplished performers from the target audience, classroom testing, end-of-course evaluation, and follow-on surveys. Evaluation results are continually reviewed and incorporated as appropriate into EMI's training program. Additionally, the Remedial Action Management Program (RAMP) is designed to resolve important management issues. All FEMA program offices, including the Readiness program, are assigned issues through RAMP and must develop action plans designed to address program shortfalls. Many issues identified in After Action reports have been assigned to the RAMP process.

Evidence: NIC Quarterly Performance Briefings; Office of Grants & Training NIMS Monitoring Report; individual State Homeland Security Strategies Reports; state enhancement plans; state investment plans; State Emergency Operations Plans review; Evaluation results.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The performance of managers is regularly evaluated in quarterly and year end performance reviews. Also, under the Department of Homeland Security MAXHR the compensation for managers will be based largely on the performance of their programs. Contractors are held accountable for performing services in accordance with documented requirements, within negotiated cost, and within prescribed timelines. In March 2005, then Department of Homeland Security Secretary, Tom Ridge issued a memorandum to all cabinet members, Agency Directors, Members of Congress, and Governors that all Federal Agencies and Departments are required to adopt NIMS and to use it in their individual emergency management programs. In September of 2004 the NIC issued a memorandum to address three requirements for the adoption and implementation of NIMS within Federal Agencies. The NIC meets regularly one-on-one and in large groups with the Federal Agencies to ensure that NIMS is implemented effectively. In FY 2007, the NIC will issue a report to the White House on the annual state of U.S. Homeland Security Preparedness within the Federal Agencies.

Evidence: FEMA's Employee Performance System; MAXHR documentation and website, and contract files.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Readiness develops its portion of the agency operating plan with quarterly allocations and executes its spending plan in a timely manner and in accordance with acquisition requirements. Funds are certified for their intended purpose. Quarterly and Mid-year reviews are conducted at the Department, Directorate, and Division levels and adjustments are made to ensure timely obligation of funds.

Evidence: FEMA's Financial Acquisition Management Division Annual Performance and Accountability Reports; Integrated Financial Management Information System (IFMIS); and EMI Quarterly Performance Briefings; NIC Quarterly Performance Briefings; NIC weekly financial reports.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Through the agency's program, planning and budgeting and strategic planning efforts, the Readiness program has developed measures to analyze current processes and procedures. FEMA has a Competitive Sourcing and Privatization Officer to which Readiness submits an annual staffing inventory. Readiness follows DHS and FEMA policies and procedures for the acquisition of IT hardware and services. EMI has also developed a cost efficiency measure to determine cost per student hour of training delivered; however, FY 2006 is the year in which baseline data will be developed to establish future measures.

Evidence: FEMA's Strategic Plan; Quarterly and Biannual Briefings; DHS, FEMA, and NETC policies and procedures regarding IT acquisitions. Office of Management and Budget (OMB) Circular A-76, Performance of Commercial Activities, dated May 29, 2003; and Department of Homeland Security Management Directive Number 0476, Performance of Commercial Activities, February 25, 2004.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Readiness program collaborates and coordinates effectively with all elements of FEMA; other DHS components; other federal agencies; State, local, and tribal officials; and Non-Governmental Organizations (NGO) as it relates to training, NIMS, and the NRP.

Evidence: Dept. of Education Emergency Response Crisis Management & Grant Program Meetings; Credentialing Discipline Groups; Resource Typing Discipline Groups; Compliance Working Group; United Southern & Eastern Tribes (USET) Meetings; pending MOU with DHS Office of Grants & Training; Preparedness Directorate. EMI conducts extensive coordination of its training program at the Federal, State, local, and tribal levels, is an active member of the DHS Training Leaders Council, and routinely works with DHS components such as the Office of Grants and Training, U.S. Fire Administration, and Customs and Border Protection, as well as other Federal agencies such as Centers for Disease Control, Veterans Affairs, Department of Defense, and the Department of Health and Human Services. EMI conducts an annual National Training and Exercise Conference to address current and emerging training and exercise issues and concerns; Dept. of Education Emergency Response Crisis Management & Grant Program Meeting; Credentialing Discipline Groups; Resource Typing Discipline Groups; United Southern & Eastern Tribes (USET) Meetings.

Does the program use strong financial management practices?

Explanation: Regular reports are run to track expenditures, to detect problems within the financial management process, and to identify resources for reprogramming or reallocation. The financial management information system is standard for all FEMA programs and has passed audits from GAO and OMB.

Evidence: Integrated Financial Management Information System (IFMIS); NIC Quarterly performance Briefings; NIC weekly financial reports; EMI Quarterly Performance Briefings; annual audits.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The NIC uses annual monitoring to revise and develop metrics to improve upon NIMS implementation. Changes and updates to the NIMS document occur as a result of establishing and maintaining advisory committees and stakeholder working groups. An integral part of the managing EMI's training program includes obtaining feedback from participants, assessing the validity of the feedback, and incorporating input as appropriate into the curriculum. The Readiness program as a result of Hurricane Katrina After action Reports that stress achieving full national NIMS compliance requires significant additional education, outreach, and training as well as certification of the nations emergency management network. The NIC is strengthening the certification process, in FY 2007 NIMS compliance will be measured with objective performance-based measures and metrics to allow assessment of the NRP and NIMS implementation status for all departments and agencies, and all state and local governments. Additionally, The Remedial Action Management Program (RAMP) is designed to resolve important management issues. All FEMA Program offices, including the Readiness Program, assigned issues through RAMP and must develop action plans designed to address the program shortfalls. Many issues identified in After Action reports have been assigned to the RAMP process.

Evidence: Training evaluation results; Ramedial Action Management Program (RAMP); Hurricane After Action Reports: White House Homeland Security Counsel's Report, Inspector General's Report; Senate Government Affairs Report; and House of Representatives Reports.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The Readiness program has demonstrated adequate progress in achieving its long-term performance goals. The NIC program has met its yearly goal of having 100% of States self-certified as NIMS compliant. EMI has exceeded its long term performance goal of annually achieving 80% in FY 2006-11 and 85% in FY 2012 of respondents reporting that they are better prepared to deal with disasters and emergencies as a result of training. This goal includes a roll-up of training activity for both EMI and FEMA's Learning Development Branch; it also includes the National Fire Academy in FY 2004. FY 2005 data reflects 84.3% and in FY 2004 83% of respondents reported that they are better prepared to respond to a disaster or emergency because of their training.

Evidence: Office of Grants & Training NIMS Monitoring Report; individual State Homeland Security Strategies Reports; state enhancement plans; state investment plans; State Emergency Operations Plans review; FYO7-12 FYHSP Report; DHS Planning, Programming, Budget, and Execution system; FEMA Budget Narratives and Quarterly Performance Reviews.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The Readiness program has achieved its annual performance goals. In FY 2005 100% of States self-certified NIMS compliance. EMI's resident training program has grown from 7,000 students in FY 2000 to over 14,000 in FY 2005. EMI's independent study (IS) program has grown from 65,000 completions in FY 2000 to over 1 million in FY 2005. This measure of students trained has a proportionate impact on enhancing the nation's emergency management capabilities. EMI data also reported in FY 2005 that 84.3% of respondents reporting that they are better prepared to deal with disasters and emergencies as a result of their training, 4.3% above the annual goal of 80%. In FY 2004 EMI data demonstrates that 83% of respondents reported they are better prepared to deal with disasters and emergencies as a result of their training, 3% above the annual/long-term goal of 80%.

Evidence: Office of Grants & Training NIMS Monitoring Report; individual State Homeland Security Strategies Reports; state enhancement plans; state investment plans; State Emergency Operations Plans review; State Emergency Operations Plans review; FYO7-12 FYHSP Report; DHS Planning, Programming, Budget, and Execution system; FEMA Budget Narratives and Quarterly Performance Reviews; NETC Admissions system for resident training statistics and EMI Independent Study Program Database for IS completions.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: EMI has developed a cost efficiency measure to determine the cost per student hour of training delivered; however, FY 2006 is the year in which baseline data will be developed to establish future measures. The NIC program is a new program that has not had the lifespan necessary to demonstrate improved efficiencies or cost effectiveness.

Evidence: DHS FYO7-12 FYHSP Report; DHS Planning, Programming, Budget, and Execution system; FEMA Budget Narratives and Quarterly Performance Reviews.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: EMI is comparable with the quality of other government training programs. It is difficult to assess whether cost efficiency is comparable since EMI is still developing baseline data and a review of available cost efficiency data for comparable federal training programs was difficult to analyze since there was not sufficient detail to determine what expenses are included in the bottom line cost. The NIMS Integration Center is not aware of any programs that have similar purpose and goals.

Evidence: DHS FYO7-12 FYHSP Report; DHS Planning, Programming, Budget, and Execution system; FEMA Budget Narratives and Quarterly Performance Reviews.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The Readiness program has utilized independent After Action Reports (AAR) following disasters as a method of evaluation for the programs effectiveness in achieving results. Recent Hurricane Katrina After Action Reports by Congress, the White House, and DHS Inspector General have all been used to review and upgrade current processes and functions within the Readiness program. As a result of the reports a full review and update will be made to the National Response Plan in FY 2007. AARs also have caused the NIC and EMI to review current educational, training, and outreach programs to incorporate report recommendations.

Evidence: Hurricane After Action Reports: White House Homeland Security Counsel's Report, Inspector General's Report; Senate Government Affairs Report; and House of Representatives Reports.