Detailed Information on the
Transportation Security Administration: Transportation Security Support Assessment

TSA will institutionalize strong program management principles and techniques to ensure accountability on behalf of contractors and staff. By the second quarter of 2007, TSA should publish policy documents and guidance to all programs, including TTAC. Furthermore, TSA will ensure staff have the proper skills to manage large capital investment programs by creating an action plan for filling skill gaps.

TSA will develop a performance management system that clearly defines and quantifies performance standards and accountability for managers. Institutional guidance will be provided to all staff on the establishment of this new system and development of personnel performance evaluation criteria.

By the end of FY2008, TSA will develop two replacement Intel measures to demonstrate the performance and efficiency of the program.

By March 17, 2007, TSA should develop and submit an aggressive plan to correct material weaknesses in financial management.

TSA will improve its budgetary program planning by establishing schedules and object classes for obligations. TSA will track this data on a quarterly basis to ensure that actual expenditures are consistent with the plan. Furthermore, TSA will develop a plan to resolve all current material weaknesses and ensure strong financial management practices.

TSA will engage GAO, DHS IG and other independent auditors to encourage, to the greatest extent possible, comprehensive evaluations of the effectiveness of the security support systems. TSA will work with the DHS IG on the development of their audit plan and request studies of this program.

TSA will establish baselines and targets for its long-term, annual and efficiency measures in the FY 2008. TSA will begin demonstrating results against these performance measures in FY 2007.

TSA will use acquisition vehicles that shift much or all of the risk away from the government, such as firm fixed price contracts. TSA will improve its contracting practices to ensure that minimum performance thresholds are documented and incentives and penalties are provides for good/poor performance.

By March 17, 2008, TSA should develop and submit an aggressive plan to correct material weaknesses in financial management.

Measure: Level of customer satisfaction with user support resolution.

Explanation:Portions of the TSS program tracks customer satisfaction metrics through follow-up SPOC field surveys that are based on a 5.0 Likert scale. These monthly metrics are a compilation of daily follow-up calls in response to user support issues. The questions include responses based on promptness, courteousness, technical service, and problem solving. Using a response of 1 as the lowest (customer dissatisfaction) and 5 as the highest (customer satisfaction), TSA is able to identify potential areas for improvement and areas where support is satisfactory.

Measure: Level of customer satisfaction with the intelligence products provided.

Explanation:This measure shows the overall level of customer satisfaction with intelligence products produced and disseminated by TSIS. This information is drawn from a survey intended for initial distribution in FY 2006, (note, a baseline is being developed during FY 2006). The survey is distributed to both internal (FSDs and FAMS) and external stakeholders (intelligence community).

Measure: Percentage of customers utilizing the TSA Remote Access to Classified Enclaves (TRACE) intelligence sharing program.

Explanation:This measure shows the percentage of targeted customers using the 206 Transportation Security Intelligence Service Remote Access to Classified Enclaves (TRACE) workstations. In FY 2004 99 of 191 workstations used the program, in FY 2005, 170 of 190. Note, FY 2005 is considered an anomaly due to the focus mandated by previous TSA leadership.

Measure: Reduce cost and schedule variance for information technology infrastructure projects.

Explanation:This efficiency measure allows both Government and contractor Program Managers (PM) to have visibility into technical, cost, and schedule planning, performance, and progress of the contract effort.

Measure: Percent decrease in workman's compensation claims.

Explanation:This measure will track improvements that have been made in reducing the costs associated with workman's compensation claims because of the high rate of on-the-job injury rates at TSA. TSA has one of the highest injury rates in government, that is why this measure is important. The workman's compensation program is managed by the Transportation Security Support program.

Measure: Percentage of systems certified based on Federal Information System Management Act (FISMA), as accepted by DHS and accredited as designated by CIO.

Explanation:This is a measure of certified IT systems. (Note: In FY 2004, TSA had only seven operational IT systems. The certification effort became significantly more challenging in FY 2005, when the number of systems spiked up to 107. This resulted in TSA missing its target. The process has stabilized and the current and future year targets are valid.) This measure is important to TSA's Transportation Security Support program because IT security is administered through this program. TSA plans to achieve their 100% by 2006 and maintaining it will be a difficult challenge as the number of IT systems changes.

Measure: Increase in percentage of reporting in the Daily Intelligence Summary directed at Terrorists groups and or activities that reflect a direct threat to the U.S. transportation system.

Explanation:Savings in people-hours and effort expended on reporting intelligence information not required (non-transportation system) for TSA decision making. Expending FTE hours on non-relevant issues, will produce more effective intel information for users (reports of relevant information/time), thus why it is an efficiency measure.

Is the program purpose clear?

Explanation: The program's purpose is to support the protection of the nation's transportation system by: receiving, assessing, and distributing intelligence information related to transportation security; providing comprehensive technical infrastructure support, i.e., information technology (IT) equipment and technical support; and providing support to accomplish the overall TSA mission.

Evidence: Aviation Transportation Security Act (ATSA) of 2001 (P.L. 107-71); Homeland Security Presidential Directive 7; DHS Management Directive 0007??IT Integration and Management; Clinger-Cohen Act of 1996; Executive Order 13388; CRS Report for Congress-Aviation Related Findings and Recommendations of the 9/11 Commission; Homeland Security Act of 2002; GAO Report (Sep 2005) TSA??More Clarity on the Authority of Federal Security Directors is Needed; February 16, 2006??Assistant Secretary Kip Hawley's Testimony before the US House of Representatives Subcommittee on Homeland Security.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program supports the operational activities of TSA, which focus on mitigating the security threat that continues to exist against the nation's transportation system. The program provides critical intelligence information, IT equipment and technical support, and administrative support; all required protect the transportation system. Also, the President's Management Agenda (PMA) is carried out by the strong leadership from these programs in the areas of strategic planning, human resource succession planning, information technology, and department-wide financial management.

Evidence: DHS IT Strategic Plan; TSA IT Strategic Plan; Homeland Security Presidential Directive 7??Critical Infrastructure Identification, Prioritization, and Protection; CRS Report for Congress??Aviation Related Findings and Recommendations of the 9/11 Commission; Homeland Security Act of 2002; GAO Report (Sep 2005) TSA??More Clarity on the Authority of Federal Security Directors is Needed; February 16, 2006?? Assistant Secretary Kip Hawley's Testimony before the US House of Representatives Subcommittee on Homeland Security; TSIS Success Stories; DHS PMA Scorecard

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program provides critical support in fulfilling the mission of securing the overall transportation system. Although DHS and other Federal Departments and agencies have intelligence programs, TSA has exclusive statutory responsibility for intelligence information related to transportation security. In addition, TSA must provide IT equipment and technical support for its unique operational activities, and it must provide the necessary administrative support to ensure the efficient and effective execution of its mission.

Evidence: GAO-04-509??Information Technology: Homeland Security Should Better Balance Need for System Integration Strategy with Spending for New and Enhanced Systems; GAO-06-598T??Homeland Security: Progress Continues, but Challenges Remain on Department's Management of Information; Secretary Chertoff's DHS Second Stage Review Remarks; GAO Report (Sep 2005) TSA??More Clarity on the Authority of Federal Security Directors is Needed; February 16, 2006??Assistant Secretary Kip Hawley's Testimony before the US House of Representatives Subcommittee on Homeland Security; TSA TRACE/RASP Project Briefing; Overview: TSIS and Information Sharing Briefing.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Due to the rushed nature of establishing TSA, there was a lack of institutionalization of information technology policies and management controls for enterprise architecture, security, capital management. TSA is updating its classified system for disseminating intelligence information to stakeholders for a higher-speed, more reliable system. Although TSA has faced many daunting challenges, TSA is addressing these issues through partnerships with other Federal and non-federal experts, and institutionalizing effective policies and management structures.

Evidence: GAO-04-509??Information Technology: Homeland Security Should Better Balance Need for System Integration Strategy with Spending for New and Enhanced Systems; GAO-06-598T?? Homeland Security: Progress Continues, but Challenges Remain on Department's Management of Information.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program's resources directly support the program's purpose to provide support to the protection of the transportation system. This evidenced by extensive headquarters reorganization that occurred in November 2005 to enhance communication and mobility of the workforce and bring TSA into shaper alignment with the newly-reorganized DHS organizational structure. Furthermore, intelligence analysts' duties are assigned based on current assessment of threat to transportation infrastructure. TSA is currently still in the process of establishing its foundational information technology architecture by deploying hi-speed systems at all locations. TSA is also in the process of developing a to-be and transition strategy for its Enterprise Architecture to better target resources for future investments.

Evidence: TSA Organization Charts - Before and after HQ reorganization; HQ Reorganization Articles in the November 2005 and January 2006 Sentinel (TSA's Newsletter) that provide comments from TSA's Assistant Secretary regarding the HQ reorganization; GAO-04-509??Information Technology: Homeland Security Should Better Balance Need for System Integration Strategy with Spending for New and Enhanced Systems; GAO-06-598T?? Homeland Security: Progress Continues, but Challenges Remain on Department's Management of Information. HLS EA 2006 Target Applications and Target Components.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has two long-term outcome goals to guide program management and budgeting and promote results and accountability. Specifically, for the Intelligence function, TSA is measuring level of customer satisfaction with intelligence products. This measure is consistent with other intelligence organization's goals that have been developed for the PART and are aligned with the Director of National Intelligence (DNI) goals, although TSA is not part of the DNI community. In addition, TSA is measuring customer satisfaction with user support provided for its information technology infrastructure.

Evidence: FY 2007 and FY 2008 TSA Congressional Justifications; TSIS Resource Allocation Plan; TSIS Program Overview; draft measures report.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: While the program has outcome-oriented long-term goals, TSA has not yet established baselines and ambitious targets and timeframes for these measures. TSA hopes to establish baselines and targets will be established at the end of FY 2006 and include them in the FY 2008 President's Budget.

Evidence: FY 2007 and FY 2008 TSA Congressional Justifications, draft measures report.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has a limited number of annual output performance measures that support the overall program objective and link to achievement of the long-term outcome measures. These goals are discrete and quantifiable; the program also has two efficiency measures to drive continuous program improvement. Specifically, a few of these goals are: percent of customers using the classified intelligence sharing system; percent of information technology systems that are certified and accredited; and percent workman's compensation injury rates. The program is best served by output measures that relate to the program's long-term outcome measures of improving customer satisfaction.

Evidence: FY 2007 and FY 2008 TSA Congressional Justifications, draft measures report.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program is developing baselines and ambitious targets for two of its four annual performance measures. TSA plans to establish baselines and targets for these measures at the end of FY 2006 for inclusion in the FY 2008 President's Budget. The program has established a baseline and targets for two of the measures -- percent of information technology systems certified and accredited and percent of customers using the intelligence sharing system.

Evidence: FY 2007 and FY 2008 TSA Congressional Justifications, draft measures report.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: This program has a number of partners that all work together toward the program goals and mission. Many of the partners are contracting companies and include; Unisys, General Dynamics, ABBTECH, SPAWAR, OpenBand, MC Dean, and USIS. The support these companies provide are aligned specifically with these goals and mission in mind. For example, for the TSA IT Managed Services contract, all tasks are documented and monitored to ensure job completeness, accuracy, quality and timeliness. In addition, it requires the contractor to track and report on variance compared to the cost, schedule and performance baseline.

Evidence: Contractor Statements of Work and detailed project schedules; TSA IT Bridge Contract HSTS03-06-D-CIO500.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Independent evaluations have been conducted on various elements of the program, but the scope has been limited and not fully directed towards TSA activities. For example, two recent Inspector General (OIG) reviews of Federal Information Security Act compliance evaluated the intelligence IT systems, but did not specific recommendations for the program. Another OIG review evaluated the management controls of the screener workforce contract, which is also just a small portion of this program. There have also been two evaluations of the entire DHS information technology program by the General Accountability Office; however, the scope focused on the department's program with no specific evaluation or recommendation of the TSA IT program.

Evidence: OIG-05-34 Report Evaluation of DHS' Security Program and Practices for its Intelligence Systems (Classified); OIG-06-13 Report DHS Has Not Implemented an Information Security Program for its Intelligence Systems (Classified); OIG-06-18 Review of the Transportation Security Administration's Management Controls Over the Screener Recruitment Program, December 2005; OIG-06-23 Transportation Security Administration's Information Technology Managed Service Contract, February 2006; GAO-04-509??Information Technology: Homeland Security Should Better Balance Need for System Integration Strategy with Spending for New and Enhanced Systems; GAO-06-598T??Homeland Security: Progress Continues, but Challenges Remain on Department's Management of Information.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Transportation Security Support's annual and long-term goals are integrated in budget requests and demonstrate how program performance will be impacted if funding levels change. The program's Congressional Justification also includes all direct and indirect program costs needed to meet its performance goals.

Evidence: FY 2007 and FY 2008 TSA Congressional Justifications; OMB 300; TSIS Resource Allocation Plan; TSIS Program Overview.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: As a result of the PART, TSA has developed long-term measures and improved annual measures. The program is in the process of developing baselines, targets, and timeframes for these measures that will support improvements in program management and increase efficiency and effectiveness. TSA will also work with the DHS Inspector General and all other independent auditors to ensure, to the extent possible, that all future audits are more comprehensive in scope and provide specific recommendations on TSA support functions.

Evidence: FY 2007 and FY 2008 TSA Congressional Justifications.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program collects timely and accurate performance data to facilitate effective management decisions in the areas of human resources, watch operations, intelligence production, and IT system effectiveness in accordance with the contracts. For example, the TSA IT Managed Services contract specifically requires the vendor to provide EVMS reporting at either the project summary level or the appropriate sub-level (phase or site) on an on-going basis. Also, TSA's partnership with the United States Coast Guard Finance Center requires reporting of productivity performance reports on timely payments. In the past, productivity reports revealed only 80% of invoices being paid on time. TSA has undertaken corrective action to fix the situation, including implementing daily monitoring mechanisms to identify invoices which were nearing the point of delinquency. As a result of these efforts, TSA's timely payment rate has moved back above 90%.

Evidence: TSA's Performance Management Information System (PMIS); DHS' Future Years Homeland Security Program (FYHSP) System; TSA IT Bridge Contract HSTS03-06-D-CIO500.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: TSA does not yet have a formalize performance management system that requires an individual's performance be clearly defined and quantified with performance measures to show who is responsible for achieving key program results. TSA is improving its acquisition vehicles to use more firm fixed price contracts. Specifically, the TSA IT managed services contract clearly stipulates what penalties will be assessed when the vendor fails to meet service level requirements and what financial rewards will be given when expectations are exceeded.

Evidence: Reports and documentation on quarterly reviews and required data; OI Statements of Work, Supplemental Performance Work Plan, and Performance Agreement for other Employees; TSA IT Strategic Plan; TSA IT Bridge Contract HSTS03-06-D-CIO500 Performance Management and Incentive Plan (PMIP).

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: TSA is unable to provide spend plans to show that the program obligates resources consistently based on a program plans, broken out by major object classes.

Evidence: Program Spend Plan, end-of-year status reports, and internal office status-of-funds documentation

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has adopted three efficiency measures, including reducing the cost and schedule variance on information technology project to no greater than 10 percent; reducing workman's compensation injury rates to address TSA's high workman's compensation claims, which is one of the highest in government; and increase the percentage of reporting directed at Terrorists groups and or activities that reflect a direct threat to the U.S. transportation system; this measure has both baseline and targets established. Furthermore, the program utilizes other mechanisms to achieve efficiencies and cost effectiveness, including the use of performance-based contracts; IT improvement plans, such as deployment of the Hi-SOC program.

Evidence: FY 2007 and FY 2008 TSA Congressional Justifications; TSA IT Bridge Contract Performance Management and Incentive Plan (PMIP); Hi-SOC program plans.

Does the program collaborate and coordinate effectively with related programs?

Explanation: This program collaborates with all appropriate partners in related programs by participating in interagency taskforces, working groups, and liaison programs. For example, Intelligence collaborates with the Central Intelligence Agency, the Federal Bureau of Investigations, the National Counter Terrorism Center, the Terrorist Screening Center and the National Security Administration by providing an Office of Intelligence FTE to work in these agencies on a daily bases. This provides for greater communication and information sharing among these intelligence entities. Furthermore, TSA collaborates with the Department on information technology projects and ensures alignment to the DHS Enterprise Architecture.

Evidence: Secretary Chertoff's DHS Second Stage Review remarks, Charter for DHS Intelligence Career Force Working Group, GAO Report (SEP 05) TSA: More Clarity on the Authority of Federal Security Directors is Needed (pgs 7 & 8); DHS Enterprise Architecture.

Does the program use strong financial management practices?

Explanation: The draft FY 2005 Audit of Financial Statements revealed three material weaknesses in the areas of financial reporting; undelivered orders, accounts, and grants payable; and budgetary accounting.

Evidence: FY 2005 Audit of Financial Statements.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: TSA is addressing its management deficiencies through a number of activities. It conducts monthly management reviews that evaluate performance metrics to evaluate progress of the transportation security support program. TSA has developed and is implementing a plan address to the FY 2003 and FY 2004 Financial Statement Audit Material Weaknesses; TSA will also develop a plan to address the FY 2005 weaknesses. TSA plans to establish baselines and targets for its efficiency measures and include them in the FY 2008 Congressional Justification.

Evidence: FY 2005 Statement of Reasonable Assurance of Achievement of Management Control Objectives. Corrective action plans for the FY 2003 and FY 2004 material weaknesses.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has not yet developed baselines and targets for its long-term outcome goals; therefore results have not been demonstrated.

Evidence: FY 2007 and FY 2008 TSA Congressional Justifications.

Does the program (including program partners) achieve its annual performance goals?

Explanation: TSA did not meet its goal to certify and accredit all its IT systems in FY 2005, per the outlined goal. TSA is establishing baselines and targets for the other measures.

Evidence: FY 2006 DHS FYHSP, FY 2007 and FY 2008 TSA Congressional Justifications.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: TSA has established baselines for two of its three efficiency goals, but has not yet demonstrated results based on actual achievement of this measure. For the other efficiency goal, TSA is current collecting baseline data.

Evidence: TSIS Report Delivery Data; IT Contract Management Data. FY 2008 TSA Congressional Justification. Draft measures report.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There is limited data available to compare components of this program with other programs at this time. TSA will work towards collecting a data in a systematic fashion that will allow a comparison of programs with similar purpose or goals. For example, the intelligence element of the program could easily be compared to CIA, FBI, DIA, etc. With the new performance measures that align to DNI goals, this should be possible.

Evidence: Current data on all programmatic elements and independent evaluations.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The limited independent evaluations by the GAO, DHS IG and DCAA indicate that the program is improving its effectiveness and achieving some results. For example, TSA is esablishing management controls and institutionalizing information technology policies for enterprise architecture, security and capital management; however, a lot of work still remains and TSA is working to address these issues.

Evidence: GAO-04-509??Information Technology: Homeland Security Should Better Balance Need for System Integration Strategy with Spending for New and Enhanced Systems; GAO-06-598T?? Homeland Security: Progress Continues, but Challenges Remain on Department's Management of Information. OIG-05-34 Report Evaluation of DHS' Security Program and Practices for its Intelligence Systems (Classified). OIG-06-13 Report DHS Has Not Implemented an Information Security Program for its Intelligence Systems (Classified).