Detailed Information on the
National Protection & Programs Division: Cyber Security Assessment

Collect data for the new performance measures to inform progress and more effective goal setting. The program will also identify additional areas for efficiency measurement.

Develop a strategic plan that outlines a strategy to address program deficiencies and cyber security challenges, and incorporate this plan into the budget request.

Establish a methodology that will ensure that resource allocation to key stakeholders is based on objective criteria such as risk, regular engagement with senior management, and its level of importance as it pertains to the Cyber Security Strategic Plan

Measure: Percent of targeted beneficiary satisfaction with cyber security collaboration events

Explanation:This measure tracks targeted beneficiaries' satisfaction with Cyber Security collaboration events, such as symposiums, cyber exercises and major forums. After collaboration events, voluntary feedback from beneficiaries on the program's performance is collected and evaluated to provide feedback.

Measure: Number of cyber security information sharing products distributed to Cyber Security stakeholders

Explanation:The program must facilitate outreach and engagement with Federal, public, private, and international entities. This measure counts (1) the number of cyber security communication and guidance products and services (including conferences conducted/sponsored, guidelines, interagency working groups, major reports, plans, speeches, training conducted/sponsored, and workshops conducted/sponsored) provided to Federal agencies; state, local and tribal governments; non-governmental organizations such as industry and academia; international entities; and individual users and (2) total number of alerts, bulletins, security tips, and vulnerability notes disseminated to stakeholders for the purpose of communicating cyber warning information in support of coordinating defenses against and responses to cyber attacks. These products provide timely information about current security issues, vulnerabilities, and exploits and patch information when available. Additionally, Cyber Security Tips offer advice about common security issues for non-technical computer users.

Measure: Percent of targeted stakeholders who have implemented the Control Systems Security Self Assessment Tool (CS2SAT) to conduct vulnerability assessments. (New measure, added February 2008)

Explanation:This measure evaluates the use of the CS2SAT tool by asset owners/operators to conduct assessments to identify and mitigate vulnerabilities in their control systems (CS). Information will be collected across CS owners/operators at the annual Process CS Forum and the International Instrumentation Symposium. Targeted stakeholders are determined based on estimated risk level of the stakeholder, stakeholder receptivity to the product, and level of impact the tool may have on stakeholder protection and prevention needs in control systems. For the public sector in 2008, the CS2SAT will target facilities managed by the Bureau of Reclamation, Army Corps of Engineers, Tennessee Valley Authority, and Bonneville Power Administration. Over time, private sector customers will be incorporated into the measure as distribution to these markets mature. Each customer has multiple facilities. The sum of these facilities serve as the denominator for target calculation.

Measure: Cost per incident (USD) reported by US-CERT

Explanation:This measures the cost per incident reported to US-CERT. A computer incident within US-CERT is, as defined by NIST Special Publication 800-61, a violation or imminent threat of violation of computer security policies, acceptable use policies, or standard computer security practices. Incidents are reported to US-CERT by Federal agencies, private sector businesses, and the public.

Is the program purpose clear?

Explanation: The Cyber Security program mission is to ensure the security, resiliency, and reliability of the Nation's cyber infrastructure in collaboration with public and private sector partners. Of particular focus is the development of capabilities to prepare for and respond to catastrophic incidents that could degrade or overwhelm the networks, systems and assets that comprise our nation's information technology infrastructure. Cyber Security will accomplish its mission by building capabilities along three strategic priorities: 1) Prepare and Deter against catastrophic incidents by achieving a collaborative risk management and deterrence capability with a mature information sharing partnership(s) between Government and the private sector; 2) Respond with a coordinated, National system to major cyber and communications disruptions to restore essential communications; 3) Build Awareness for a well-informed public--at home and in the enterprise--who understands the shared responsibility we all have to protect our part of the cyber and communications network; Cyber Security's mission and strategic priorities closely align to the goals set forth within the Department of Homeland Security's Strategic Plan.

Evidence: National Strategy for Homeland Security (2002), Homeland Security Act of 2002, The National Strategy to Secure Cyberspace (2003), National Infrastructure Protection Plan (2006), The National Strategy for the Physical Protection of Critical Infrastructure and Key Assets (2003), Executive Order 13286, Homeland Security Presidential Directive - 7, FISMA Section 3546, National Response Plan, Cyber Security Program Management Review, Cyber Security Strategic Plans

Does the program address a specific and existing problem, interest, or need?

Explanation: In response to 9/11, it was determined a single integration point was necessary to: 1) coordinate and partner with the private sector, government, military and intelligence stakeholders in risk assessment and mitigation of vulnerabilities and threats to critical IT assets and functions that affect the operation of US critical infrastructures; and 2) provide cyber threat and vulnerability analysis, early warning, and incident response assistance for public and private sector constituents. Cyber Security, in its unique coordination and collaboration role among the public and private sectors, serves as the national focal point required for these activities. The threat of cyber attacks and the vulnerabilities to our nation's infrastructure remain real, and the mission of Cyber Security remains more relevant than ever before. Although improvements in securing America's cyberspace have been made since Cyber Security's inception (such as expanded reach and increased awareness to stakeholders, increased collaboration efforts among private and public sector organizations, and improved vulnerability identification methods), the threat of cyber attacks continues to grow. For example, the US-CERT received over 23,000 incident reports in FY2006, an increase from 4,700 incidents in FY2005. In addition, over 7,000 new vulnerabilities were identified in FY2006 as compared to 4,800 vulnerabilities identified in FY2005.

Evidence: National Strategy for Homeland Security (2002), Homeland Security Act of 2002, The National Strategy to Secure Cyberspace (2003), National Infrastructure Protection Plan(2006), The National Strategy for the Physical Protection of Critical Infrastructure and Key Assets (2003), Executive order 13286, Homeland Security Presidential Directive - 7, FISMA Section 3546, National Response Plan, Cyber Security Program Management Review, Cyber Security Strategic Plans, GAO-06-1087t, GAO-06-672, GAO-05-434, and GAO-04-354

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: While many individual organizations in the public and private sectors strive to secure their parts of the cyber infrastructure, the role to coordinate and lead among the stakeholders remains clearly and solely Cyber Security's. Cyber Security designs and operates a number of cyber security programs and collaborates with federal, state, local, private and international entities to coordinate and direct resources effectively without duplication of efforts. While these entites hold some similar responsibilities within their specific domain or sector areas, Cyber Security coordinates and prioritizes activities among these complementary efforts. As a collaboration lead, Cyber Security collects information from its many partners and leverages that information to drive standards, awareness, and capabilities. No other federal agency partners with federal, state, and local communities, international partners, and private sector industry in securing the nation's cyber space with such a broad scope. For example, the FBI's cyber mission is focused on the most serious criminal computer intrusions and spread of malicious code; identifying and thwarting online sexual predators who prey on children and distribute child pornography; counteracting operations that target U.S. intellectual property; and dismantling organized criminal enterprises engaged in Internet fraud. The U.S. Secret Service cyber mission is more narrowly focused on protecting the nation from computer based attacks on the nation's financial, banking, and telecommunications infrastructure. Specific programs that illustrate Cyber Security's unique leadership role include: 1) US-CERT Operations Center. Provides a national coordination center that links public and private capabilities to facilitate information sharing across all infrastructure sectors and to help protect and maintain the continuity of our nation's cyber infrastructure 24x7x365. 2) Einstein Program. An automated process for collecting, correlating, analyzing, and sharing computer security information across the Federal civilian government. 3) Critical Infrastructure Protection-Cyber Security Program. Strengthens preparedness by partnering with public and private sectors to improve the security of the IT Sector, as well as cyber security across all of the Nation's critical infrastructure sectors in implementing the National Infrastructure Protection Plan (NIPP).

Evidence: National Strategy for Homeland Security (2002), Homeland Security Act of 2002, The National Strategy to Secure Cyberspace (2003), National Infrastructure Protection Plan (2006), The National Strategy for the Physical Protection of Critical Infrastructure and Key Assets (2003), Executive Order 13286, Homeland Security Presidential Directive - 7, FISMA Section 3546, National Response Plan, Cyber Security Program Management Review, Cyber Security Strategic Plans, Future Years Homeland Security Program (FYHSP)

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Several external reports have indicated that the Cyber Security program does contain some flaws that limit its strategic and operational effectiveness. Cyber Security acknowledges that it has room for organizational improvements that would strengthen the program's effectiveness and efficiency in responding to both major and minor cyber events. Evaluations by GAO and DHS IG indicate that Cyber Security is not adequately achieving its strategic goals. The new senior leadership is actively attempting to address these concerns. For example, Cyber Security is in the process of revising its strategic plan, which will tie the efforts of individual programs back to the goals identified in the strategic plan and to budgeting activities. These revisions will similarly be supported by the implementation of new, more robust performance measures.

Evidence: The National Strategy to Secure Cyberspace (2003), Cyber Security Program Management Review, Cyber Security Strategic Plans, GAO-06-1087t, GAO-06-672, GAO-05-434, and GAO-04-354

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Cyber Security currently has not institutionalized a method for tying beneficiaries to resource allocation. The program is evaluating how to distribute and prioritize resources across key external stakeholders and is actively working with its program components to outline how the various program area resources are/should be allocated to internal and external Cyber Security stakeholders. Cyber Security intends to establish a methodology that will ensure that resource allocation to key stakeholders is based on objective criteria such as risk, regular engagement with senior management, and its level of importance as it pertains to the Cyber Security Strategic Plan. Cyber Security is actively working with Program Managers to define criteria for prioritization that will enhance opportunities for synergy across the whole organization. Until this methodology is institutionalized, Cyber Security will not be able to adequately demonstrate that its resources are effectively targeted.

Evidence: The National Strategy to Secure Cyberspace (2003), National Response Plan, Cyber Security Program Management Review, Cyber Security Strategic Plans

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Cyber Security has added a new long term outcome measure - Percent of targeted beneficiary satisfaction with Cyber Security collaboration events. This new measure will capture the satisfaction level of targeted participants with events, such as symposiums, cyber exercises, and major forums provided by the Cyber Security program. At the conclusion of a Cyber Security event, voluntary feedback that asks beneficiaries to determine whether they were or were not satisfied with an event's outcome, will be solicited. This data will then be evaluated to determine the overall beneficiary satisfaction with that event. By tracking this data, Cyber Security will be able to evaluate the usefulness of its current products to its beneficiaries and apply this knowledge to future programmatic improvements. This performance measure addresses the following goals from the Cyber Security and Communications (CS&C) strategic plan: 1) Prepare and Deter against catastrophic incidents by achieving a collaborative risk management and deterrence capability with a mature information sharing partnership(s) between Government and the private sector; 2) Respond with a coordinated, National system to major cyber and communications disruptions to restore essential communications; 3) Build Awareness for a well-informed public--at home and in the enterprise--who understands the shared responsibility we all have to protect our part of the cyber and communications network

Evidence: Performance Measures Definition and Data Collection Form, Cyber Security Strategic Plans, FYHSP, Contractor Monthly Status Reports, Program Management Review, Performance Metrics Implementation Plan, IT Sector Specific Plan Metrics and Milestones, Advance Acquisition Plan, Monthly Status of Funds Report, expectmore.gov

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Cyber Security recently proposed a new long-term, outcome performance measure - Percent of targeted beneficiary satisfaction with cyber security collaboration events. Because this is a new performance measure, no actual data on which to base reliable targets currently exists. The program plans to implement data collection, beginning in June of 2007. As Cyber Security continues to gather this data for this measure, the program will evaluate feasibility and aggressiveness of the targets.

Evidence: Performance Measures Definition and Data Collection Form, Cyber Security Strategic Plans, FYHSP, , Program Management Review, Performance Metrics Implementation Plan

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Cyber Security has proposed five new annual performance measures that will demonstrate progress towards achieving the Cyber Security mission and development of strategic priorities. 1. The number of alerts, bulletins, security tips, and vulnerability notes provided to stakeholders as part of the National Cyber Alert System. To accomplish its mission, Cyber Security has identified a need to address cyber attacks by coordinating incident response activities through the dissemination of alerts, bulletins, security tips and vulnerability notes. These activities directly support Cyber Security's mission to provide cyber threat and vulnerability analysis, early warning, and incident response assistance to public and private sector constituents. 2. The number of cyber security communication and guidance products and services provided to stakeholders. Actions tracked in this measure improve the security, resilience and reliability of the Nation's infrastructure by enabling the sharing and development of assets that enable and strengthen collaboration with the public, private, and international entities to secure cyberspace and America's cyber assets. 3. The number of potentially malicious incidents identified by US-CERT/EINSTEIN. US-CERT/EINSTEIN identifies and analyzes suspicious network activity across the Federal government, in an effort to ensure the security, resiliency, and reliability of the Nation's cyber infrastructure. By quantifying the number of malicious and anomalous incidents identified by EINSTEIN, the measure reflects the efficacy of the program's objective to encourage federal agencies to develop and implement security controls that address and mitigate risk. 4. The number of stakeholders using the Control Systems Cyber Security Self Assessment Tool (CS2SAT). CS2SAT is a tool aimed at reducing the risk of cyber attacks on facilities and control systems/assets by identifying vulnerabilities within industrial control system architectures. CS2SAT provides industrial control systems with a repeatable and systematic approach to assessing the cyber security posture of industrial control system networks, a comprehensive evaluation and comparison to existing industry standards and regulations, and raises the awareness of control system cyber security within facilities. 5. The cost per incident (USD) reported to and analyzed by US-CERT. This measure will reflect improving operating efficiencies within the US-CERT program by calculating the total cost associated with the services provided when responding to incidents reported by US-CERT.

Evidence: Performance Measures Definition and Data Collection Form, Cyber Security Strategic Plans, FYHSP, Contractor Monthly Status Reports, Program Management Review, Performance Metrics Implementation Plan, IT Sector Specific Plan Metrics and Milestones, Advance Acquisition Plan, Monthly Status of Funds Report

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Cyber Security recently proposed five new annual output and efficiency performance measures. Because these are new performance measures, there is little actual data on which to base reliable targets. Cyber Security has collected some verifiable data for four of the new measures. Baselines for three of its measures have been created using data collected from June 2006 through May 2007 ([1] The number of cyber security communication and guidance products and services provided to stakeholders; [2] The number of alerts, bulletins, security tips, and vulnerability notes provided to stakeholders as part of the National Cyber Alert System; and [3] The cost per incident reported to and analyzed by US-CERT); and the baseline for another (The number of potentially malicious incidents identified by US-CERT/EINSTEIN) was derived using data collected from September of 2006 through May 2007. One of Cyber Security's new annual measures currently has no actual data on which to base reliable targets. (Number of stakeholders using the Control Systems Cyber Security Self Assessment Tool). The program plans to implement data collection beginning in June of 2007. As Cyber Security continues to gather data for this measure, the program will evaluate feasibility and aggressiveness of the targets.

Evidence: Performance Measures Definition and Data Collection Form, Cyber Security Strategic Plans, FYHSP, Program Management Review, Performance Metrics Implementation Plan

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Cyber Security partners are important to the success of the program. Cyber Security has worked closely with federal partners (e.g., NIST, DoD, and DOJ), private sector industry leaders (e.g., IT Sector Coordinating Council, National Cyber Security Alliance), and internal DHS organizations (e.g., Office of Infrastructure Protection, Science and Technology Directorate, and National Communications System). Cyber Security's long-term and annual program metrics take into account the performance of its partners such as through work products created. Public and private sector partners are committed to Cyber Security performance levels through signed Memorandums of Understanding, such as those within the EINSTEIN program. As a result, this past year installations were completed of Einstein at seven volunteer pilot Departments and agencies MOUs are in place for partners in the EINSTEIN program, Scholarship for Service Program, Centers of Academic Excellence in Information Assurance Education, and Interagency Agreements are in place with NIST, SPAWAR, and Carnegie Mellon. Cyber Security also works with Information Sharing and Analysis Centers (ISACs), from the IT and other industries, in efforts to reduce risk such as through the implementation of the IT Sector Specific Plan and the NIPP. The IT Sector Specific Plan defines agreed goals, associated milestones, and roles and responsibilities for the protection of the sector. In addition, contractual and financial controls are in place to make sure contractor partners meet performance obligations. Cyber Security works closely with its COTRs to ensure projects are completed on time, within budget, and products produced of good quality. Monthly status reports are also reviewed to provide regular checks to ensure contractor performance is on track.

Evidence: Performance Measures Definition and Data Collection Form, FYHSP, Contractor Monthly Status Reports, Program Management Review, Performance Metrics Implementation Plan, Memorandums of Understanding, IT Sector Specific Plan Metrics and Milestones, Monthly Status of Funds Report

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Multiple evaluations have been conducted by the GAO in 2006 and 2007 focusing on the Cyber Security's progress toward securing cyber infrastructure. [DHS Leadership Needed to Enhance Cyber Security (GAO-06-1087T); DHS Faces Challenges in Developing a Joint Public/Private Recovery Plan (GAO-06-672); DHS Faces Challenges in Fulfilling Cyber Security Responsibilities (AO-05-434); Critical Infrastructure Protection: Challenges and Efforts to Secure Control Systems (GAO-04-354)] The audits reviewed Cyber Security's overall performance in achieving its mission as well as highlighting several specific areas for improvement. For example, reducing the control system risk within and across critical infrastructure sectors is a Cyber Security priority. GAO report 06-1087T examined Cyber Security's effectiveness in forming partnerships with the private sector and other government agencies to conduct a comprehensive cyber risk assessments in terms of cyber vulnerabilities and threat assessments. The GAO recommended that Cyber Security develop and implement a coordination strategy to improve control system security, including an approach for coordinating the various ongoing efforts to secure control systems. In response, Cyber security created the Control Systems Security Program (CSSP). The CSSP seeks to reduce control system risks within and across all critical infrastructure sectors. The program coordinates efforts among federal, state, local, and tribal governments, as well as control systems owners, operators and vendors.

Evidence: GAO reports, IG reports, GAO-06-1087t, GAO-06-672, GAO-05-434, and GAO-04-354

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Currently Cyber Security does not adequately utilize performance data to support and justify budget requests and resources are not succinctly tied to the accomplishment of Cyber Security's long-term and annual goals. Cyber Security is actively engaged in developing the necessary mechanisms that would specifically tie the successful accomplishment of program performance goals to the budgeting process. These mechanisms include the development of more robust performance measures that will be specifically tied to strategic plans and goals, as well as incorporating existing measures into the budget process.

Evidence: Performance Measures Definition and Data Collection Form, Advance Acquisition Plan (AAP), Program Management Report, Integrated Planning Guidance, FYHSP, Congressional Budget Submission and Justification, OMB Exhibit 300s

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Cyber Security has not adequately addressed deficiencies in the program's strategic planning activities. Currently, Cyber Security is in the process of actively reviewing and updating its strategic plan for relevancy and strategic alignment with the DHS and Cyber Security & Communications (CS&C) goals and objectives, both short and long-term.

Evidence: Cyber Security Strategic Plans, S1 Report on CS&C, GAO response

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Cyber Security has found it challenging to define measures that thoroughly capture results reflecting the program's mission. The program has not institutionalized the use of performance data in program management activities. Recently, the program has been collecting data for four of its new annual measures for at least nine months. From the data collected so far, Cyber Security is beginning to find areas for improvement. For example, data collected for the EINSTEIN output measure reflects the importance of continuing to support the implementation of EINSTEIN software sensors. As a result, Cyber Security is continuing to upgrade its EINSTEIN software sensor network which will allow Cyber Security to be able to identify and analyze suspicious network activity across the Federal Government.

Evidence: DHS Integrated Planning Guidance, Peer Reviews, Program Management Report, FYHSP, After Action Reports

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Cyber Security collaborates across its program areas to ensure that program personnel are held accountable for evaluating and improving upon the effectiveness and efficiency on matters related to cost, schedule, and performance execution. Cyber Security uses the DHS Performance Appraisal System (PAS) to rate the performance of its personnel at all levels. Each individual performance plan includes line items that detail the performance requirements of the manager including details on individual goals and budgets that tie to those of the program. Reviews are held twice a year. In addition, incentives are in place for those that meet and exceed their goals and criteria. All personnel performance information is tracked and kept within the DHS PAS tool. Partners of the program are held to their commitments through contracts and deliverables, and their performance evaluated monthly via monthly status reports. In addition, monthly Contracting Officer Technical Representative (COTR) meetings allow reviews of contract performance. Release of funds to partners does not occur until contractual performance criteria are met. While NCSD has not experienced poor performance from contractors, any contractors with repeated deviations from the prescribed statements of work and timelines will be documented and addressed immediately by the COTRs. Regular oversight meetings with partners help ensure they are performing according to Statements of Work. Relationships with other private and public sector partners are formalized and enforced through Memorandums of Understanding (MOUs) and Memorandums of Agreement (MOAs). In situations where there are issues with compliance to signed agreements, letters of compliance are distributed to the program's partners. In addition, if any risks are identified, the federal program managers are directly held as the key points of contact to address and remediate risks to ensure partners meet and are held accountable for the performance and cost schedules.

Evidence: : Integrated Planning Guidance, Cyber Security Strategic Plans, MOUs, Contract Monthly Status Reports, Program Management Review, Performance Metrics Implementation Plan, MAXhr, Memorandum of Agreements

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Cyber Security continues to obligate funds on schedule in accordance with their intended purposes. Cyber Security financial obligations are tracked to vendor payment schedules in the Federal Financial Management System (FFMS). Financial reviews are held on a weekly to quarterly basis among program area budget analysts and program managers to ensure prompt and proper obligation and management of funds occurs. All budget execution activities comply with the Federal Financial Management Integrity Act and follow the Generally Accepted Accounting Principles (GAAP). The program maintains appropriate execution of obligation rates. Monthly budget reporting among program managers, annual reconciliation efforts, mid-year reviews, and management controls all ensure the appropriate use and obligation of funds are in accordance with its intended purposes. Cyber Security has historically demonstrated successful obligation of its funds as reflected in the program's 100% obligation rate in FY05 funds and 99.9% obligation rate in FY06. The FY07 obligation targets have been set, and the program has been successful in meeting its quarterly targets and is on track in meeting its annual obligation target.

Evidence: FYHSP, DHS Integrated Planning Guidance, OMB Circular A11, Program Management Review, MaxPortal, Monthly and Quarterly Federal Financial Management System Reports, Cyber Security Status of Funds Report, Congressional Budget Submission, Contract Monthly Status Report

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Currently Cyber Security has no active measures used to determine the efficiency of program execution. Cyber Security recently submitted a new efficiency measure (Cost per incident reported to and analyzed by US-CERT) that will reflect the effectiveness of the financial resources applied to address cyber incidents, by dividing the total costs associated with incident handling by the total number of incidents for the reporting period.

Evidence: FYHSP, DHS Integrated Planning Guidance, Program Management Review, monthly and quarterly Federal Financial Management System Reports, Cyber Security Status of Funds Report, Advance Acquisition Plan, Contractor Monthly Status Reports, Performance Metrics Implementation Plan

Does the program collaborate and coordinate effectively with related programs?

Explanation: Cyber Security serves as the national center of excellence for cyber security programs. Cyber Security products leverage the information of its partners and are therefore dependent upon effective relationships and collaboration with its partners. The program's commitment to effective collaboration and coordination with its external and internal DHS partners is evidenced by the number of strategic collaborations and working groups such as through the EINSTEIN program, ISACs, Government Forum of Incident Response and Security Teams (GFIRST) and Chief Information Security Officer (CISO) Forum. For, example, one of Cyber Security's successful collaboration and coordination roles is the Information Systems Security Line of Business (ISS LOB) Program. In 2006, the Office of Management and Budget (OMB) established the ISS LOB to support the President's Management Agenda goal to expand Electronic Government. OMB designated Cyber Security as the managing agency to work with D/As and provide leadership and direction for improving information systems security services across the Federal government. The objective of the ISS LOB is to achieve more consistent security management processes and controls through the re-use of proven best practices and by promoting savings through reduced duplication and economies of scale for common hardware, software, and shared services through the establishment of Federal shared service centers. As a result, Cyber Security created an ISS LOB Program Management Office to ensure that the government's IT security programs include comprehensive and consistently implemented risk-based, cost-effective controls. These enable and support vulnerability reduction through end-to-end shared services, business improvement solutions, and common and repeatable tools and processes that are available to all the federal D/As.

Evidence: Cyber Security Strategic Plans, Performance Metrics Implementation Plan, MOUs, GAO response, After Action Reports, IT Sector Specific Plan, Contract Monthly Status Report

Does the program use strong financial management practices?

Explanation: Cyber Security is fully compliant with all Federal and DHS agency specific financial management regulations and policies and those standard operating policies and procedures. Cyber Security ensures that program managers are held accountable for evaluating the effectiveness and efficiency of their responsible areas on matters related to cost, schedule, and performance. Strong financial management processes are in place and certified through compliance with the Federal Financial Management Improvement Act (FFMIA). These compliance requirements call for management controls to ensure resources are allocated effectively to avoid fraud and mismanagement. Centrally-managed funds are managed by a set of strong financial controls for recording, processing, and/or reporting, which includes a chain of reviews by managers to ensure expenditures support the program's goals and approved spending plans. All budget activities comply with the DHS Integrated Planning Guidance, FYHSP, FFMS, OMB Circular A11, and Government Performance and Results Act. Regular meetings and reviews also occur to ensure compliance to internal financial management standards such as the monthly COTR meetings to ensure contractual requirements are met and monthly budget reports to ensure programmatic financial targets are met.

Evidence: DHS Integrated Planning Guidance, Federal Financial Management System, OMB Circular A 11, FYHSP, Cyber Security Monthly Status of Funds Report, Government Performance and Results Act

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Cyber Security is actively engaged in the ongoing improvement of its program management activities. The program is enhancing internal controls, strengthening methods for the validation and verification of data, taking steps to improve contract management activities and cost tracking, and is engaging in Earned Value Management of its programs. Cyber Security is also tracking its progress towards remedying deficiencies as identified by external evaluators.

Evidence: GAO responses, Program Management Review, Performance Metrics Implementation Plan

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Because no actual data exists for the new long-term performance measure, at this time Cyber Security cannot demonstrate adequate progress in achieving long term performance goals. Cyber Security will begin data collection on the new outcome measure to ensure it can demonstrate that it is achieving its long-term performance goals.

Evidence: Cyber Security Strategic Plan, FYHSP, Program Management Review, Performance Metrics Implementation Plan

Does the program (including program partners) achieve its annual performance goals?

Explanation: Because sufficient data has not been collected for the new annual performance measures, at this time Cyber Security cannot demonstrate adequate progress in achieving annual performance goals. Cyber Security will begin data collection on all new annual measures to ensure it can demonstrate that annual performance goals are met and illustrate progress towards its long-term goals.

Evidence: Cyber Security Strategic Plans, FYHSP, Program Management Review, Performance Metrics Implementation Plan

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Because sufficient data has not been collected for its new efficiency measures, at this time Cyber Security cannot demonstrate adequate progress in achieving annual goals. Cyber Security will begin data collection to ensure it can demonstrate the needed efficiencies across all programs.

Evidence: Performance Measures Definition and Data Collection Form, Cyber Security Strategic Plans, FYHSP, Program Management Review, Performance Metrics Implementation Plan

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There are no other programs that perform the same role as the Cyber Security program to offer a similar function for comparison. Cyber Security leads and coordinates public and private sector partner activities to protect the Nation's cyber infrastructure. Although outside evaluations have been performed with operational advice for improvements, no analysis has led to alternative programs or means by which to achieve Cyber Security's mission. Cyber Security has succeeded in defining an expansive reach and footprint for its products and services and moved closer to accomplishing its national mission that addresses federal, state, local, private, and international segments. In order to illustrate the value of the Cyber Security, an outcome performance measure has been developed that assesses the stakeholder satisfaction as it pertains to the various Cyber Security products and services delivered such as cyber exercises, major forums, symposiums, and collaborative efforts with various stakeholders.

Evidence: National Strategy for Homeland Security (2002), Homeland Security Act of 2002, The National Strategy to Secure Cyberspace (2003), National Infrastructure Protection Plan (2006), The National Strategy for the Physical Protection of Critical Infrastructure and Key Assets (2003), Executive Order 13286, Homeland Security Presidential Directive - 7, FISMA Section 3546, National Response Plan, Cyber Security Program Management Review, Cyber Security Strategic Plans

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The external audits of Cyber Security by GAO have found that the program has significant room for improvement. Cyber Security is currently tracking its progress towards remedying deficiencies as identified by external evaluators.

Evidence: GAO reports, IG reports