Detailed Information on the
Transportation Security Administration: Transportation Security Officer Workforce Assessment

Annual Efficiency Measures: Revise the costs per passenger and baggage screened measures so that each measure only factors in Transportation Security Officer costs directly used for screening passenger or baggage by October 31, 2007 and revise out year targets accordingly by the end of Q1 FY08.

System Design Vulnerabilities: Develop a covert testing program comparable to the Aviation Screening Assessment Program (ASAP) that will measure the effectiveness of alternative baggage screening procedures and incorporate testing of secondary baggage screening in ASAP by Q3 FY08.

Screening Standard Operating Procedure: Prepare an evaluation and piloting plan to determine how changing the Prohibited Items List will impact the effectiveness of passenger screening by Q2 FY08, initiate the evaluation at the beginning of Q3 FY08, assess preliminary results in Q1 FY09, and implement system-wide SOP updates at the beginning of Q2 FY09.

Career Enhancement: Enhance career growth and professional development opportunities for the TSO workforce by increasing the number of FTEs in specialized technical career tracks such as the Aviation Direct Access Security (ADAPS) Program, Behavioral Detection Officers ( BDOs), and Bomb Appraisal Officers (BAOs). Assess preliminary results Q4 FY09.

Long-term Outcome Measures: Revise the passenger screening and baggage screening covert test results measures per the new Aviation Screening Assessment Program (ASAP) by the end of Q4 FY07 and develop new out year targets based on results from the April-to-September ASAP pilot by October 15, 2007.

Resource Allocation: Improve the Screener Allocation Model to facilitate sensitivity analyses by increments of one minute for the FY08 cycle by the end of Q4 FY07 and develop a plan to increase the accuracy of the model through field validation of key assumptions and system performance data by Q2 FY08 in preparation for the FY09 cycle.

Measure: Passenger screening covert test results

Explanation:This measures the percentage of the time that Transportation Security Officers (TSO) correctly identify prohibited material at passenger checkpoints during covert tests in order to reduce the probability of a successful terrorist or other criminal attack to the air transportation system. The target and actual results are classified for security reasons and are not releasable to the public at this time.

Measure: Baggage screening covert test results

Explanation:This measures the percentage of the time that Transportation Security Officers (TSO) screening baggage correctly identify prohibited material during covert tests in order to reduce the probability of a successful terrorist or other criminal attack to the air transportation system. The target and actual results are classified for security reasons and are not releasable to the public at this time.

Measure: Level of the Customer Satisfaction Index (CSI-A) for Aviation Operations

Explanation:The CSI-A is a composite index incorporating data on security confidence, passenger surveys, and compliments/complaint data on screener performance. TSA aspires to provide effective screening operations with minimum disruption to the traveling public. CSI-A is scaled where 0 is very dissatisfied, 25 is dissatisfied, 75 is satisfied, and 100 is very satisfied.

Measure: Cost per passenger screened

Explanation:This measure will suggest the system-wide human resources cost of screening a person based on the total salary and benefits costs of the TSO workforce and the number of people who pass completely through walk-through-metal-detectors at airport screening checkpoints.

Measure: Cost per bag screened

Explanation:This measure will suggest the system-wide human resources cost of screening a bag based on activity based costing information at a sample of airports.

Measure: Percentage of screeners scoring above the national standard level of Threat Image Projection (TIP) performance

Explanation:Transportation Security Officers (TSOs) must be proficient in using scanning equipment in order to safeguard the public against terrorist and criminal attacks on the air transportation system. TSA established a standard level of TIP performance, and the measure reflects the percentage of screeners performing above the standard. TSOs receive ongoing training and performance assessments to ensure that their skills are being developed to address the variety of threats that may be presented. As threats change and evolve, the TIP program develops new images and training to address the expanded needs of the TSO workforce, allowing TSA to maintain a high level of screener performance that ensures aviation security.

Is the program purpose clear?

Explanation: The Transportation Security Administration (TSA) Transportation Security Officer (TSO) Workforce program is the certified security screening personnel stationed at over 450 airports across the U.S. responsible for performing pre-board security screening of persons and their carry-on and checked baggage, protecting the traveling public by preventing any deadly or dangerous objects from being transported onto an aircraft.

Evidence: Section 110(a) of the Aviation and Transportation Security Act (ATSA) of 2001 requires that TSA provide for the screening of all passengers and property, including U.S. mail, cargo, carry-on and checked baggage, and other articles, that will be carried aboard a passenger aircraft operated by an air carrier or foreign air carrier in air transportation or interstate air transportation...

Does the program address a specific and existing problem, interest, or need?

Explanation: Intelligence collected by the United States Government continues to identify a significant, ongoing security threat to the nation's transportation systems, as evidenced most recently by the August 2006 plot in the United Kingdom to attack U.S. commercial aviation with liquid explosives. As aviation remains a focus of terrorist plots, TSA has evolved the responsiveness and effectiveness of the Transportation Security Officer (TSO) Workforce to shifting security threats through intensive training in threat item awareness, use of advanced screening technology, and the ability to detect suspicious behavior. TSOs have found more than 40 million prohibited items (knives, explosives, guns, weapons, etc.) at screening checkpoints and prevented them from getting on planes. Each day, TSA intercepts more than 15,000 prohibited items at airports around the country. Each month, more than 40 firearms are intercepted at airport checkpoints by TSOs.

Evidence: 1. FY 2008 TSA Congressional Justification. 2. February 13, 2007 Written Statement of Assistant Secretary Kip Hawley before the House Subcommittee on Homeland Security, Committee on Appropriations. 3. 3-1-1 for Carry-Ons. 4. Prohibited Items List. 5. Remarks by Assistant Secretary of Homeland Security for the Transportation Security Administration Kip Hawley to the International Air Transport Association Conference on October 17, 2006 in Sydney, Australia. 6. Remarks by Homeland Security Secretary Michael Chertoff, United States Attorney General Alberto Gonzales, FBI Director Robert Mueller, and Assistant Secretary for TSA Kip Hawley - August 10, 2006. 7. Classified TSA threat intelligence.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: As mandated by ATSA, as a result of the horrible attacks on September 11, 2001, TSA created a new system for Federal employees to screen all passengers and baggage before boarding commercial passenger aircraft. Although ATSA mandated the federalization of airport security screening, it held open the possibility that airports could return to contract screening, provided the high standards required by law and instituted by TSA are met. Beginning in 2002, ATSA required TSA to conduct a contract screening pilot program (known as the "PP5") at five airports to evaluate the performance of private screeners relative to federal screeners. In 2004, TSA established the Screening Partnership Program (SPP) to meet the requirement for the "opt-out" program established in ATSA allowing airport operators to apply to have screening functions performed by qualified private screening contractors. However, TSA remains responsible for overseeing security standards and managing contractor performance.

Evidence: 1. Aviation and Transportation Security Act of 2001 (ATSA). 2. Screening Partnership Program Quality Assurance and Surveillance Plan and Award Fee Plan. 3. February 13, 2007 Written Statement of Assistant Secretary Kip Hawley before the House Subcommittee on Homeland Security, Committee on Appropriations.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The TSO Workforce Program is designed free of major flaws. TSA is right-sizing and stabilizing screening operations based on security requirements and opportunities for increasing effectiveness and efficiencies. As part of its TSO Workforce planning, TSA implemented several initiatives: Local Hiring; Screening Allocation Model (SAM); Dual-Function TSOs; and Optimization and Safety Teams. TSA uses the Screening Allocation Model (SAM) to determine the level of staffing each airport needs to operate at maximum efficiency and effectiveness. Federal Security Directors (FSDs) have more hiring authority at the local level to address airport staffing needs. TSA's Standard Operating Procedures (SOPs) for screening passengers and checked baggage were modified to permit FSDs to deploy TSOs to conduct screening functions in both passenger and checked baggage screening locations. TSA created an integrated Optimization and Safety Team to conduct in-depth airport site visits and identify ways to improve the efficiency and effectiveness of local screening operations and safety programs.

Evidence: 1. Screening Allocation Model (SAM) Briefing. 2. GAO Report 07-299. 3. Screening Management Standard Operating Procedures (Sensitive Security Information (SSI)). 4. OD 400-34-5: Optimization and Safety Team Airport Visits- Roles & Responsibilities.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Expenditures for the TSO Workforce directly support the program's core purpose, screening passengers and carry-on and checked baggage, and thus directly reach the intended beneficiaries on a daily basis. During 2006, TSOs screened over 708 million passengers with an average wait time of 3.79 minutes; screened over 535 million individual pieces of checked luggage; opened 16 percent of checked bags (over 85 million bags) searching for prohibited items; and intercepted over 13 million prohibited items at security checkpoints, which included 1.6 million knives.

Evidence: 1. Section 110(a) of the Aviation and Transportation Security Act of 2001. 2. FY 2008 TSA Congressional Justification. 3. Screening Statistics, Facts and Figures for 2006.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: TSA has three long-term measures for its TSO Workforce Program - Passenger Screening Effectiveness, Baggage Screening Effectiveness, and the Customer Satisfaction Index for Aviation Operations - that focus on long-term outcome performance and meaningfully measure the ability of the TSO Workforce to protect commercial air travelers by preventing dangerous or deadly objects that individuals are attempting to smuggle onto an aircraft either on themselves or in their carry-on or checked baggage.

Evidence: 1. FY 2008 TSA Congressional Justification 2. FYHSP

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The TSO Workforce program has adequately ambitious targets for its long-term goals that promote continuous improvement of current capabilities. Baselines and targets are established with the Program, adjusting as factors change over time.??

Evidence: 1. FY 2008 TSA Congressional Justification 2. FYHSP 3. FY 2006 DHS Performance and Accountability Report

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has three measures that reflect the performance of important aspects??of the??program - accuracy in screening and cost effectiveness.?? They are discrete and quantifiable and help drive continous program improvement of the workforce.?? The effectiveness of screeners is measured through an output measure that reflects the percentage of screeners performing above the TSA national standard level for identifying threat items through scanning equipment. The two other efficiency measures track the average personnel costs to TSA per passenger screened and baggage screened.

Evidence: 1. FY 2008 TSA Congressional Justification 2. FYHSP 3. FY 2006 DHS Performance and Accountability Report 4. TSA Management Review/Office of Security Operations Monthly Objectives Report

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has developed baselines for all measures and ambitious targets for one of its annual performance measures, which is also updated on a regular basis depending on actual data and operational changes that affect the program and the measure. However, the targets for its two annual measures are not adequately ambitious enough.

Evidence: 1. FY 2008 TSA Congressional Justification 2. FYHSP 3. FY 2006 DHS Performance and Accountability Report 4. TSA Management Review/Office of Security Operations Monthly Objectives Report

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: With the help of its many partners, TSA has created a new system that is dramatically different from that which was in place on September 11, 2001. In order for TSA to be effective in carrying out its aviation security duties, it is important to build and maintain effective relations with all partners.?? To ensure all partners are working toward the same goal as TSA in protecting America against terrorism, TSA spends a great deal of time with its partners and providing them with security information.?? Specific activities include meeting regularly with airport managers and security directors to discuss security needs and directives, developing coordinated command and control systems with emergency response agencies, coordinating law enforcement activity in support of aviation security, collaborating with airlines to identify and resolve issues that impact efficient passenger flow and customer service while maintaining security standards, and working with airport and airline managers to identify upcoming changes in demand for passenger and baggage screening.?? TSA has held press conferences at many airports around the country to educate passengers about requirements and restrictions, including the prohibited items list. ??TSA also prominently posts signs in airports to help passengers understand which items are prohibited, and detailed information is provided on??its website at www.tsa.gov . ?? TSA works to build a foundation of trust and confidence with the American people and their Congressional representatives through respecting individual privacy and having the TSO Workforce perform its mission with a high level of customer service.

Evidence: 1. Screening Partnership Program Quality Assurance and Surveillance Plan and Award Fee Plan 2. February 13, 2007 Written Statement of Assistant Secretary Kip Hawley before the House Subcommittee on Homeland Security, Committee on Appropriations

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: This program is assessed on an ongoing basis through comprehensive independent audits and evaluations of sufficient scope and quality by the DHS Office of Inspector General (OIG) and the U.S. Government Accountability Office (GAO). The DHS OIG and the GAO have released several reports recommending improvements to TSA management of various aspects of the TSO Workforce program. TSA has used these recommendations to implement improvements to the program.

Evidence: DHS OIG Reports: OIG-07-14; OIG-07-04; OIG-06-65; OIG-06-18; OIG-06-10; OIG-05-51; OIG-05-17. GAO Reports: GAO-07-448T; GAO-07-299; GAO-06-371T

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The TSO Workforce Program's annual and long-term goals are integrated in budget requests and demonstrates how program performance will be impacted if funding levels change. In addition, the Congressional Justification includes all program costs needed to meet its performance goals.

Evidence: 1. FY 2008 TSA Congressional Justification 2. FY 2008 Performance Budget Overview 3. FYHSP

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: As a result of the 2004 PART, TSA has developed performance goals, and established long-term and annual performance measures with baselines and targets that support improvements in program management and increase efficiency and effectiveness.

Evidence: 1. FY 2008 TSA Congressional Justification 2. TSA TSO Workforce PART Improvement Plan 3. FYHSP

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: TSA regularly collects timely and credible performance data from key program partners to manage the program, adjust priorities, allocate resources, and improve performance. This includes customer satisfaction information and confiscation of prohibited items to allow for comparison and analysis. Another example of TSA using effective management decision to improve screening operating, is implementing the Screening Allocation Model (SAM) to determine the level of staffing each airport needs to operate at maximum efficiency and effectiveness. Additionally, under the Screening Partnership Program (SPP), TSA collects data on private screening operations to ensure compliance with security standards and performance requirements.

Evidence: 1. FYHSP. 2. TSA Management Review/Office of Security Operations Monthly Objectives Report. 3. Screening Partnership Program Quality Assurance and Surveillance Plan and Award Fee Plan. 4. February 13, 2007 Written Statement of Assistant Secretary Kip Hawley before the House Subcommittee on Homeland Security, Committee on Appropriations.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: FSDs are responsible for oversight of TSO Workforce operations at each federalized airport, as well as overseeing security standards and managing contractor performance at the six contract screening airports under SPP. FSDs evaluate performance of TSOs using a four-level system called the Performance Accountability and Standards System (PASS). The objective of PASS is to promote and sustain a culture of high performance and accountability tied to TSA's annual and long-term goals. Additionally, TSA contracts under the Screening Partnership Program ensure accountability for cost, schedule and performance.

Evidence: 1. TSA Management Directive 1100.43-1: Performance Accountability and Standards System. 2. Screening Partnership Program Quality Assurance and Surveillance Plan and Award Fee Plan.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Program funds are obligated consistently to budget request and program objectives. TSA has execution reports to show the program obligates resources consistently based on the program's plans. Most of the TSO Workforce program spending relate to payroll expenditures as opposed to acquisition expenditures. TSA's financial controls over payroll have improved considerably over recent years.

Evidence: 1. FY 2008 TSA Congressional Justification. 2. DHS/TSA Monthly Execution Report.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: TSA manages the execution of resources in accordance with its overall program objectives, with DHS and TSA financial management directives, and with reporting measures that ensure the program is being executed effectively and efficiently. The TSO Workforce program has efficiency measures with baselines and targets, such as cost per passenger screened and cost per baggage screened. Additionally, other mechanisms include an integrated Optimization and Safety Team to visit airports and implement improvements in screening efficiency, safety programs, staffing models, and related activities.

Evidence: 1. FY 2008 TSA Congressional Justification. 2. FYHSP. 3. TSA Financial Management Manual. 4. TSA Management Review/Office of Security Operations Monthly Objectives Report. 5. OD 400-34-5: Optimization and Safety Team Airport Visits- Roles and Responsibilities. 6. Optimization and Safety Briefing, January 2007.

Does the program collaborate and coordinate effectively with related programs?

Explanation: TSA closely coordinates the activities of the TSO Workforce Program with all Federal, state, and local and programs that have a role in protecting aviation security in the United States. TSA partners include aircraft operators; the Federal Air Marshal Service, the Federal Flight Deck Officer program, the Federal Aviation Administration, the Federal Bureau of Investigation, and the Department of Defense for in-flight security; Joint Terrorism Task Forces; and local, state, and federal law enforcement agencies who respond to incidents at screening checkpoints. TSA also provide oversight over the private screening operations under the Screening Partnership Program.

Evidence: 1. Screening Management Standard Operating Procedures (Sensitive Security Information (SSI)). 2. TSA Security Directive - SD 1544-01-20E & SD 1544-01-21F (SSI). 3. Aircraft Operators Standard Security Plan (SSI). 4. Screening Partnership Program Quality Assurance and Surveillance Plan and Award Fee Plan. 5. February 13, 2007 Written Statement of Assistant Secretary Kip Hawley before the House Subcommittee on Homeland Security, Committee on Appropriations.

Does the program use strong financial management practices?

Explanation: TSA implemented a financial and management control program that will strengthen its ability to provide decision makers with reliable information that complies with federal accounting standards. Independent external auditors have identified material weaknesses, but none of those are specific to the TSO Workforce program, and the weaknesses do not prevent the program from effectively managing its resources. Most of the TSO Workforce program spending relate to payroll expenditures, as opposed to acquisition expenditures.

Evidence: 1. FY 2006 Audit of Financial Statements. 2. TSA Management Control Policy Statement. 3. TSA MD 3100.3 Management Control Program. 4. TSA MD 3100.3.1 Management Control Council.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: TSA has taken meaningful steps to address its management deficiencies. Corrective Action Plans are developed for audit weaknesses. Independent external auditors have identified agency-wide material weaknesses, but none of those are specific to the TSO Workforce program.

Evidence: 1. FY 2006 Audit of Financial Statements. 2. TSA FY 2007 Corrective Action Plans. 3. FY 2007 DHS Internal Control over Financial Reporting (ICOFR) playbook.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The TSO Workforce program has shown progress since 2005 in achieving its long-term performance goals. In addition, the performance goals are continually reassessed to ensure they are ambitious against the long-term progress of the program.

Evidence: 1. FYHSP 2. FY 2008 Congressional Justification

Does the program (including program partners) achieve its annual performance goals?

Explanation: The TSO Workforce program has shown some progress since 2005 in achieving its annual performance goals. TSA's personnel cost per passenger screened dropped by 23.5% between 2005 and 2006, and its personnel cost per baggage screened dropped by 52.3% during the same period. However, this does not neccessarily indicate increased efficiency but rather indicate an increase in the number of air travellers and the subsequent reduction in unused TSO screening capacity during that time. However, TSA's percentage of TSOs scoring above the TIP national standard level significantly increased since 2005.

Evidence: 1. FYHSP 2. FY 2008 Congressional Justification

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: TSA has established baselines and targets for its efficiency goals and has demonstrated results based on the actual data in meeting targets. The TSA personnel cost per person screened dropped by 23.5% between 2005 and 2006. During the same time period, the TSA personnel cost per baggage screened dropped by 52.3%. In addition, the program uses its resources efficiently to achieve results, which is greatly aided by continuously improving its use of the Screener Allocation Model to better allocate screeners at our nation's numerous airports. Furthermore, TSA implemented a new TSO Career Progression program that seeks to add significant additional security within current budget while enabling widespread career growth, professional development opportunities, and better trained TSOs. As part of the Career Progression program, TSA added specialized technical career tracks, including Behavior Detection Officers (BDO) and Bomb Appraisal Officers (BAO) who are taken "off the line" from screening operations and placed in other parts of the airport to assist screening.

Evidence: 1. Screener Allocation Model (SAM) 2. FY 2008 Congressional Justification

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There are no comparable programs .

Evidence: .

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations by the DHS OIG and the GAO have affirmed that the program is achieving results and making progress.?? However, areas for improvement have been identified,??such as the need to institutionalize the use of more rigourous analytical methodologies to determine resource allocation based on threat and the use of more rigorous testing in operational environments for all screening procedures.?? For example, the GAO identified??that??TSA has not been conducting methodologically sound evaluations to determine how changing the prohibited items list will impact the effectiveness of??its passenger screening program to detect threat items.?? As for checked??baggage screening, although TSA has a system for testing the effectiveness of??standard screening procedures--through covert and unplanned tests--such a system does not exist for alternative screening procedures which are activated in certain situations at airports.?? As a result, challenges still remain in accurately assessing the performance level of all TSA screening processes.

Evidence: GAO-07-623R; GAO-06-869; GAO-07-448T; GAO-07-299; GAO-06-371T; GAO-06-597T. Other DHS OIG Reports: OIG-07-14; OIG-07-04; OIG-06-65; OIG-06-18; OIG-06-10; OIG-05-51; OIG-05-17.