Detailed Information on the
Homeland Security Operations Center Assessment

Reduce security issues and expand user base of the current HSIN program.

Develop an outcome measure for OPS.

Reduce security issues and expand user base of the current HSIN program.

Re-evaluate performance measures and develop replacements, where necessary, of sufficient scope and quality to measure program outcomes, outputs, and efficiency.

Continue to improve upon existing measures and develop operationally focused metrics.

Measure: Cost per node or beneficiary connected to the HSOC.

Explanation:HSOC will establish the per capita cost of each user of its products and will strive to consistently reduce per user costs as total user base increases without a degradation of the other performance measures.

Measure: Actual numbers of Federal, State and local agencies connected to HSIN.

Explanation:

Measure: Percentage of Federal, State and local agencies that maintain connectivity with the Homeland Security Operations Center (HSOC) via Homeland Security Information Network (HSIN) and participate in information sharing and collaboration concerning infrastructure status, potential threat and incident management information.

Explanation:The information created by HSOC is only useful if it reaches its targeted audience. HSOC will measure the number of federal, state, tribal, and private sector partners it establishes and maintains as members of its user community. Performance will be measured as a percentage of the total target audience.

Is the program purpose clear?

Explanation: The Department of Homeland Security (DHS) was established for the primary mission of preventing terrorist attacks in the United States, reducing vulnerability to such attacks, minimizing the damage to critical infrastructure, and assisting in the response and recovery from such acts. In support of the DHS mission, The Homeland Security Operations Center (HSOC) is the primary national-level hub for operational communications and information pertaining to domestic incident management.

Evidence: "Homeland Security Presidential Directives 5, 6,7, 8, 10, 11, & 13 HSOC Concept of Operations Document Homeland Security Act National Response Plan"

Does the program address a specific and existing problem, interest, or need?

Explanation: The Presidential and Congressional actions resulting from the terrorist attacks on 9/11 identified a need for improved information sharing, collaboration, and fusion across the federal, state, and local levels both vertically and horozontally. Additionally it was determined that the US needed to be more effective in the fusion of intelligence informaton with operational information in order to detect, prevent, and deter terrorist incidents. Furthermore, it a need was determined for a single point of integration of domestic situational awareness in responding to domestic incidents and natural disasters. The HSOC is the national level hub for operational communications and information sharing pertaining to domestic incident management across Emergency Operations Centers (EOCs) at the Federal, state, Local, Private and Public Domain. The HSOC serves as the Nation's single point of threat information integration and dissemination to secure the Homeland.

Evidence: "See: HSPD-5 Homeland Security Act 9/11 Commission Report National Critical Infrastructure Plan National Response Plan"

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The National Response Plan assigns the HSOC the sole responsibility of integrading, maintaining, and disseminating daily domestic situational awareness to help detect, prevent, and deter terrorist incidents. In addition to its incident management, situational awareness, and information sharing responsibilities, the HSOC integrates information from other command and operations centers.

Evidence: "Excerpts from the Congressional Budget Submission Naitonal Response Plan DHS Command Centers Study"

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The HSOC is coordinating closely with the Offices of State and Local Coordination, Infrastructure Protection, and Information Sharing and Collaboration to ensue that the Homeland Security operations procedures are destined to serve all stakeholders. DHS is currently conducting an internal review to see if the HSOC model is free of major flaws.

Evidence: HSOC is working directly several DHS organizations (Office of State and local Coordination, Office of the Chief Information Officer, Office of Information Sharing and Collaboration) to ensure that all stakeholders requirements are accounted for in organizational and system design. This wide coordination effort will mitigate the potential for unnecessary overlap with similar programs.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: HSOC resources are being allocated to acquire and operate the systems, infrastructure, and personell to increase the quality of SA by improving the speed and integrity of the processing of information into a Common Operational Picture (COP), thus inproving the speed and accuracy of decisions and response. "HSOC provides a comprehensive and accurate picture of the current homeland threat environment. This picture provides the Situational Awareness (SA) necessary to support decision criteria for the prevention of and recovery from incidents. Additionally HSOC resources provide for the sharing and collaboration of this information via the Homeland Security Information Network (HSIN). "

Evidence: "The HSOC program plan provides a mechanism for the creation of this situational awareness (Common Operational Picture (COP)) and the vehicle for collecting the information required to create SA as well as the distribution of the SA (Homeland Security Information Network (HSIN)). Supporting documentation: -HSOC ConOps -HSOC Strategic Plan -HSOC/HSIN OMB Exhibit 300s -FY07 Congressional Justification Documents -Critical Sector ConOps -Critical Sector End User Agreements -Critical Sector Governance -HSIN & LWC connectivity in plac. The scale will be in place to accomodate new users. Stakeholders are identified in coordination with: -Office of Sate and Local Coordination -Office of Infrastructure Protection -Office of Information Analysis -IStaff and the Interagency Incident Management Group -National Response Plan"

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: HSOC has established measures that quantify the timeliness, quality, and dependability of the information it creates and shares with its beneficiaries. While these predominantly output measures are sufficient given the HSOC's level of maturity, long-term measures with targets still need to developed.

Evidence: "Congressional Budget Submission. Annual Performance Plans and Reports. Future Years Homeland Security Program. FYHSP (2006)"

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: HSOC will develop long-term measures that are ambitious and have targets that demonstrate continuous improvement.

Evidence: "Congressional Budget Submission. Annual Performance Plans and Reports. Future Years Homeland Security Program Annual Performance Plans and Reports HSOC 5 Year Objectives and Milestones."

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The HSOC's Annual Measure - Cost per node or beneficiary connected to the HSOC - demonstrates progress towards achieving the program's long-term goals.

Evidence: "Annual Performance Plans and Reports. See Measures Tab"

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The HSOC will work to develop baselines targets that directly support achieving its performance goals.

Evidence:  

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Each contract action has a set deliverable which direcly supports the goals and objectives of the year. Additionally, HSOC has separate goals and objectives which directly support relationships, MOUs, and operating procedures between HSOC and its partners (Federal, State, Local, Tribal, and Private Sectors).

Evidence: The HSOC floor represents the cooperation of over 35 federal, state & local agencies, which are jointly engaged in the accomplishment of the HSOC mission to provide comprehenve situational awareness to decision makers at all levels.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: HSOC has been evaluated by the Office of the Inspector General several times for performance and operations and security practices. HSOC utilizes independent government agencies from the Departments of Defense and Energy to preform evaluation of their policies, practices, and operations.

Evidence: "Command Center Working Group Study Certification and Accreditation by the Office of the Chief Information Officer"

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: HSOC's acquisitions strategy establishes investments to increase HSOC capabilities. Each year's budget provides for an increase in capability. These investments directly support the HSOC's annual goals and directives.

Evidence: "Congressional Budget Submission. HSOC Strategic Plan HSOC's Baselines in Capability HSOC/HSIN OMB Exhibit 300s HSOC Annual Spend Plan"

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The HSOC began operations in 2004. The original HSOC Strategic Plan required multiyear funding to achieve capability. As more data becomes available, a detailed program evaluation can be conducted. DHS is currently conducting an internal review to look at ways to improve HSOC's strategic planning functions.

Evidence:  

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: HSOC currently collects data on the total users of its products, the timeliness of data processing, and the cost to maintain and support the systems it has acquired. This data directly supports several of its performance measures. While this type of information is sufficient - given the HSOC's current level of development - for now, it is expected HSOC will implement processes to capture and apply higher-quality information as the program matures.

Evidence: "HSOC Senior Watch Officer Log HSIN User Community Data HSIN Help Desk Call Logs The program managers routinely review this data and utilize it to identify capability needs and priortization of goals and objectives. HSOC is allocating resources towards development of automated systems to reduce the cycle time of processing threat and incident information and to distribute alerts and warnings. (ie. Mega Log, HSIN, COP) The Director of the HSOC uses this information when working with partner leadership to make adjustments in interagency data flow processes and staffing levels."

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: HSOC uses a combination of Firm-Fixed-Price, Performance Based, and Cost+ contracts. Firm-Fixed-Price contracts minimize risk to cost overruns. Earn Value Management is utilized to measure the performance of contractors by assigning a quantifiable value to each milestone.

Evidence: "OMB Exhibit 300s "

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: HSOC is working to improve the obligation of funds to ensure priority investments are funded. HSOC as begun scheduling acquisitions at the beginning of each year and conducting quarterly reviews to assure program is on target. Each acquisition will allocated against the program management plan.

Evidence:  

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Acquisitions utilize competitive awards or competitivly awarded indifinite delivery/indefinite quantity (IDIQ) contracts. HSOC has enlisted the assistance of a DOD agency to assist in the analysis and planning of IT acquisitions in order to assure best practices are being followed.

Evidence: "Interagency Agreement with US Navy SPAWAR This directly supports performance measure 5, monitoring the per capita cost of connecting to our stakeholders. OMB Exhibit 300 -----------------------------------------"

Does the program collaborate and coordinate effectively with related programs?

Explanation: Given the HSOC's current level of maturity, establishing MOUs is sufficient in demonstrating progress. While the HSOC has partnered with over 35 other federal and DHS programs in order to leverage existing technology and capabilities in achieving its desired outcome, more works remains te done. DHS is working to improve the level of coordination with all partners.

Evidence: "Utilization of the IAIP Critical Warnings and Information Networks (CWIN) Program for providing connectivity to the state Emergency Operations Centers (EOC) eliminating duplication of resources. Utilization of the Joint Regional Information Exchange System (JRIES) for achieving initial capability for the HSIN. Utilization of the Defense Department's Global Command and Control System (GCCS) for achieving initial capability for its COP."

Does the program use strong financial management practices?

Explanation: Personnel are in place to do internal controls, who have current federal financial training to include COTR/COR, and Defense Acquisition training. All financial obligations are strictly monitored before payment of invoices. Accurate current records are maintained to document fund execution at every state (appropriation, commitment, obligation invoicing, and expenditure) for all obligations.

Evidence: The HSOC financial team has conducted internal execution reviews. IAIP has not yet been subject to an IG or GAO audit.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: HSOC has begun addressing management deficiencies and expects to make progress on all financial and day-to-day management issues. HSOC began operations in 2004 and has begun conducting internal program evaluations.

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The HSOC has achieved success in connecting to is partners, developing sound SA, and managing domestic events. However, the HSOC is developing long-term performance measures with target to measure its progress.

Evidence:  

Does the program (including program partners) achieve its annual performance goals?

Explanation: HSOC has achieves some if its goals and objectives over the last 2 years. All 50 states and major urban area are connected to the HSOC.

Evidence: "Participation by its partner agencies is evidence of their commitment in achieving HSOC's performance goals. Avhieving annual baselines in capibility."

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: This performance measure uses the total nodes achieved on the HSIN as a factor in calculating the per capita cost. The roll out schedule has been impacted in the first part of the year due to scheduling constraints with the States. Therefore the total number of nodes is slightly behind schedule. HSOC continues to work towards achieving it efficiency goal of reducing its per capita cost and expects to show better progress as schedule delays are overcome.

Evidence: Annual Program Performance Review Monthly / Quarterly Performance Reports (Appendix A)

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: HSOC operations has established itself as the Nation's incident management center.

Evidence: "HSOC has achieved effectiveness in a very short timeframe. Compare to similar multiagency organizations such as National Military Command Center, National Counter Terrorism Center, Federal Emergency Management Agency. No other agency has achieved the level of participation, operational capability, and nation relavance in an equal timeframe and resource investment. As the national center for incident management, HSOC has been assigned the leadership role for coordinating all federal agencies in the execution of the federal incident management plan."

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The HSOC has been audited by the Office of the Inspector General and has been reviewed by the Office of Information and Collaboration. HSOC also participates in an annual national exercise which establishes criteria for the response and recovery of multiple national incidents.

Evidence: "Top Off 2 Assessment Report Certification and Accreditation by the Office of the Chief Information Officer"