Detailed Information on the
Transportation Security Administration: Passenger & Baggage Screening Technologies Assessment

Collaborate with the Office of Security Operations to synchronize the methodologies for computing the Cost per Passenger Screened and Cost per Bag Screened measures between the Transportation Security Officer Workforce and the Passenger & Baggage Screening Technologies programs.

Determine the level of public confidence in the ability of passenger and baggage screening technology to keep air travel secure and use the data to develop new outcome measures.

Collaborate with the Office of Security Operations and others to enhance the current Passenger and Baggage Security Screening Assessment Results measures and to develop and implement a technology-specific aviation screening assessment result outcome measure.

Measure: Ratio of Passengers Screened to Passenger Screener Full Time Equivalent (FTE) Expended

Explanation:This measure is an indicator of how well TSA is performing in terms of increasing the operational capacity of a single screener with technology. These data include the number of units processed and the number of screener FTE expended during a single year. Formula = # Of Units Screened In Total During A Fiscal Year / # Of Screener FTE expended During the Same Period

Measure: Ratio of Bags Screened to Baggage Screener Full Time Equivalent (FTE) Expended

Explanation:This measure is an indicator of how well TSA is performing in terms of increasing the operational capacity of a single screener with technology. These data include the number of units processed (# of bags screened by EDS combined with # of bags screened by ETD) and the number of screener FTE expended during a Fiscal year. Formula = # Of Units Screened In Total During A Fiscal Year / # Of Screener FTE expended During the Same Period

Measure: % Increase in the Time Passenger Screening Equipment is Operational

Explanation:This measure provides an indication of the increase in the time passenger screening equipment is operational and available for use. These increases build upon the new baseline of 95% for the annual measure Percentage of Time Passenger Screening Equipment is Operational. These data include aggregated nationwide screening system uptime and downtime statistics across multiple years. Formula = ((Current Average Operational Availability (Ao) for Screening Equipment - Baseline Average Operational Availability (Ao) for Screening Equipment) / Baseline Average Operational Availability (Ao) for Screening Equipment) + Baseline % Increase In Passenger Screening Equipment Ao

Measure: % Increase in the Time Baggage Screening Equipment is Operational

Explanation:This measure provides an indication of the increase in the time baggage screening equipment is operational and available for use. These increases build upon the baseline of 98.14% for the annual measure % of Time Baggage Screening Equipment is Operational. The data include included in this measure's computation include the aggregated nationwide screening system uptime and downtime statistics across multiple years. Formula = ((Current Average Operational Availability (Ao) for Screening Equipment - Baseline Average Operational Availability (Ao) for Screening Equipment) / Baseline Average Operational Availability (Ao) for Screening Equipment) + Baseline % Increase In Baggage Screening Equipment Ao.

Measure: % of Time Passenger Screening Equipment is Operational

Explanation:This measure is an indicator of airport readiness in terms of machine operational availability. These data include aggregated nationwide screening system uptime and downtime statistics for a single year. Formula = Total uptime fleet wide / Total uptime plus total downtime fleet wide (%). NOTE: The target reduction between FY07 and FY08 coincides with a change in the Operational Availability (Ao) formula.

Measure: % of Time Baggage Screening Equipment is Operational

Explanation:This measure is an indicator of airport readiness in terms of machine operational availability. These data include aggregated nationwide screening system uptime and downtime statistics for a single year. Formula = Total uptime fleet wide / Total uptime plus total downtime fleet wide (%).

Measure: Cost Per Passenger Screened

Explanation:Formerly called Level of Machine Efficiency, this measure is an indicator of the cost per unit screened. These data include cost data from across multiple years and units processed data from a single year. Costs included in this equation are the depreciation and maintenance of equipment. Formula = (Amortization of equipment capital acquisition costs from previous and current Fiscal Year) / # Of Units Processed In Total During The Current Fiscal Year

Measure: Cost Per Bag Screened

Explanation:Formerly called Level of Machine Efficiency, this measure is an indicator of the cost per unit screened. These data include cost data from across multiple years and units processed data from a single year. Costs included in this equation are: personnel benefits costs; workforce operation costs; maintenance and utilities costs; and capital costs for the purchase in installation of equipment. Formula = Total Cost of Baggage Screening Service / Total Checked Bags Screened

Is the program purpose clear?

Explanation: The Passenger & Baggage Screening Technologies Program safeguards the Nation's aviation system by providing efficient and effective security technology solutions through applied research, development, operational testing, deployment, and life cycle management to ensure the free movement of people and commerce. This high-performing, nimble program develops and fields appropriate security technology by consistently applying proven program management principles and procedures, continuously prioritizing activities and solutions based on risk, while effectively communicating with our stakeholders.

Evidence: Evidence: Section 110(a) of the Aviation and Transportation Security Act of 2001 - Public Law 107-71; Passenger Screening Program Mission Need Statement signed April 11, 2005; the Aviation Checked Baggage Mission Needs Statement signed March 30, 2004.

Does the program address a specific and existing problem, interest, or need?

Explanation: Aviation remains one of the primary focuses of Middle East and other terrorist organizations for actions against U.S. citizens, and the airport passenger screening function constitutes the front lines of preventing aircraft contraband that can assist in terrorist and other criminal acts intended to harm passengers, aircraft and other persons and property.

Evidence: Transportation Security Administration transportation risk assessments and audits, classified intelligence/threat data collections and reports, security oversight inspections, checkpoint arrests, and dangerous item confiscation levels at airports. A world wide example that involved the USA occurred when the British police, acting with MI5, arrested 21 individuals early August 10, 2006 and disrupted an advanced terrorist plot that would have caused considerable loss of life. The 21 arrested were alleged to have engaged in a plot to detonate liquid explosives aboard multiple commercial aircraft departing from the United Kingdom (UK) and bound for the U.S.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: This program is mandated to be the only effort that screens passengers, carry-on and checked baggage for firearms, explosives, and other dangerous weapons, before boarding commercial passenger aircraft. No other civilian or government agency has authority to perform this effort since the terrorist acts on September 11, 2001. Prior to implementation of the Aviation Transportation Security Act of 2001, passenger and baggage security screening was performed by the airports and airlines in accordance with regulations set by the Federal Aviation Administration (FAA).

Evidence: Congress passed new laws establishing this program after September 2001 terrorist event. Section 110(a) of the Aviation and Transportation Security Act of 2001 requires that TSA: ''shall provide for the screening of all passengers and property, including United States mail, cargo, carry-on and checked baggage, and other articles, that will by an air carrier or foreign air carrier in air transportation or intrastate air transportation..." In addition, Section 110(d) provided that explosives detection systems should be deployed "to screen all checked baggage".

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Passenger and Baggage Screening Technologies Program works with DHS Science and Technology (S&T) for national research and development (R&D) on security technologies. In the event that S&T identifies an effective emergent technology or product, the program conducts an industry review to identify suitable sources and, when applicable, initiates the acquisition process in accordance with the Federal Acquisition Regulation (FAR). The Passenger Screening Program (PSP) recently underwent a program review during which DHS experts approved the program noting some minor deficiencies that were subsequently corrected. Additionally, the Electronic Baggage Screening Program (EBSP) has stabilized the designs of its baggage screening equipment - specifically the Explosive Detection Systems (EDS) and Explosive Trace Detection (ETD) suite of equipment. The equipment must pass specific testing that certifies the equipment as 100% effective in detection. The equipment also has a self-diagnostic test that determines if the machine is operating at 100% each morning. If this self diagnostic determines that it is not operating at 100%, then it will take itself off line until a technician can correct the issue. Furthermore, the DHS Inspector General's OIG-08-25 performance report deemed the passenger and baggage screening processes as "strong" and provided some suggestions for additional effectiveness. As a result, the following initiatives have been and/or are currently being implemented: the establishment of the Aviation Screening Assessment Program which systemically accesses program effectiveness and provides data analytics; the acquisition of advanced x-ray systems with automated detection, and the deployment of enhanced screening checkpoints that focus on hostile intent identification, robust explosive detection, and congestion reduction.

Evidence: There is no evidence that another approach is more effective or efficient

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The Passenger and Baggage Screening Technologies Program is directly and effectively targeting the traveling public, the air carriers, and the airport authorities as its beneficiaries. The primary mission to prevent the entry of firearms, explosives and other dangerous weapons on commercial aircraft, while ensuring freedom of movement for people and commerce is being met by maintaining effective legacy systems and upgrading the levels. New additions to the systems such as bottled liquids scanners, reduced sized EDS, and AT systems have begun being deployed to airports across the nation. These systems provide additional security benefits to the traveling public, airlines, and airport authorities by minimizing the ability of terrorists to submit prohibited items and explosives into the aviation network. Other beneficiaries include the Transportation Security Officers (TSO), the TSA, and the Federal Government. Creating a safer environment for the TSO to work in will reduce on-the-job injuries (OJI) and thus minimizing operating costs.

Evidence: The program is effectively targeted per the following: Review and Oversight of Program Goals and Objectives: The program goes through a regular review process at TSA that includes program and stakeholder risks and issues: ?? Annual or as necessary program progress reviews before the TSA Investment Review Board (IRB) to include the Assistant Secretary for Management ?? Quarterly evaluations of technical effectiveness and program efficiency at the Director level ?? Weekly internal program project status meetings between the Program Manager and all the Project Managers ?? Weekly Integrated Project Team meetings between the Project Managers and stakeholders ?? Additionally, the TSA Chief Technology Officer (CTO) reviews program progress directly with the Program Manager as necessary to ensure program objectives are being accomplished as aligned with TSA and DHS strategic requirements. Since the program has been initiated, no terrorists' incidents have occurred on any commercial airline that has adversely affected the public. The TSO OJI rates have dropped by more than 50% since program inception. Also, the Program Management Plan and associated Communications Plan document the review process; whereas, the mission statement and MNS define the focus.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Passenger and Baggage Screening Technology Program has two long-term measures that focus upon outcomes and meaningfully reflect the program's purpose, which is to provide efficient and effective security technology solutions through applied research, development, operational testing, deployment, and lifecycle management to ensure free movement of people and commerce. These measures include Percentage Increase in the Time Passenger Screening Equipment is Operational and Percentage Increase in the Time Baggage Screening Equipment is Operational. These measures relate directly to the lifecycle management clause in the program's statement of purpose because they are indicators of how well the TSA's current inventory of deployed screening equipment is being maintained.

Evidence: TSA OST increased the number of in-line BHS by 41 terminals (thereby getting rid of airport lobby machines and introducing scalable systems). The program is also introducing high speed systems in FY 2009. In addition, the program has continued to modify certification standards on a perpetual basis, since 2001. (The specific goals are SSI)

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program does have ambitions targets and timeframes for the long-term measures described in the response to question 2.1. For example, the Percentage Increase in the Time Passenger Screening Equipment is Operational measure provides an indication of the percentage change in the time passenger screening equipment is operational and available for use. The target increases from .75% to 1.81% between FY08 and FY11. These increases build upon the new baseline of 95% for the annual measure Percentage of Time Passenger Screening Equipment is Operational. The Percentage Increase in the Time Baggage Screening Equipment is Operational measures acts as a similar indicator. In this instance, however, the increases build upon the baseline of 98.14% for the annual measure Percentage of Time Baggage Screening Equipment is Operational. Targets for this measure rise from 1.16% to 1.37% between FY08 and FY11. These targets are ambitious, because the measures account for hundreds of pieces of technology from San Juan, Puerto Rico to the Saipan in the Northern Mariana Islands.

Evidence: See Passenger and Baggage Screening Technology Long-Term Outcome Measures and Targets.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program does have several annual measures that demonstrate short-term progress toward the achievement of its long-term goals and that relate to the program's purpose, which is to provide efficient and effective security technology solutions through applied research, development, operational testing, deployment, and lifecycle management to ensure free movement of people and commerce. These measures include the Percentage of Time Passenger Screening Equipment is Operational and the Percentage of Time Baggage Screening Equipment is Operational, which relate directly to the lifecycle management clause in the program's statement of purpose, because they are annual indicators of how well the TSA's current inventory of deployed screening equipment is being maintained. They also include the Cost Per Passenger Screened; the Cost Per Bag Screened, the Ratio of Passengers Screened to Passenger Screener Full Time Equivalents (FTE) Expended; and the Ratio of Bags Screened to Baggage Screener FTE Expended measures that show progress towards improved efficiency, which is also captured in the program's purpose statement. Annual Output and Efficiency measures have been chosen as indicators of the program's performance towards its long-term goals in lieu of Annual Outcome measures because they enable the program to show demonstrative improvement to the effectiveness and efficiency of the screening processes through the research, development and testing of new screening technologies in accordance with Section 137 of the Aviation and Transportation Security Act (ATSA).

Evidence: The program has identified annual measures to support its long term measures, goals and objectives. Some examples include: installing at least 10 in-line BHS per year, reducing the cost per bag by two cents per year, and installing next generation technology at a majority of airport terminals.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program does have baselines and ambitious targets for its annual measures. For example, the Ratio of Passengers Screened to Passenger Screener Full Time Equivalent (FTE) Expended measure is targeted to increase from 28,200 in FY08 to 31,760 in FY14. In addition, the Ratio of Bags Screened to Baggage Screener FTE Expended is targeted to increase from 40,060 to 44,513 over the same period of time. These targets are ambitious, because these increases in the ratio of screeners to units processed are expected to occur even as airline passenger volumes continue to increase. The target for the Percentage of Time Baggage Screening Technology is Operational measure increase from 98.1% in FY07 to 98.6% in FY14. Similarly, the targets for the Percentage of Time Passenger Screening Equipment is Operational measure increases from 95% in FY08 to 96% in FY14. These targets are ambitious because of the number of pieces of equipment; their geographical dispersion; the variety of operating conditions in terms of passenger volumes and climates and the complexity of the technology itself. Other examples include the Cost Per Passenger Screened and Cost Per Bag Screened measures, which are targeted to decrease from $0.12 a passenger and $3.35 a bag in FY08 to $0.08 and $3.15 respectively in FY14 despite inflation.

Evidence: See Passenger and Baggage Screening Technology Annual Output Measures and Targets.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All partners work together to meet the long term goals of the Passenger & Baggage Screening Technologies Program. The annual and long-term goals of the program are aligned with all partners such as original equipment manufacturers (OEM), maintenance contractors, support contractors, general contractors, airport authorities, airlines, Transportation Security Officer's (TSO), etc. The statements of work, task and delivery orders, and schedules of the contracts that support the program have direct input into achievement of program goals. All of the partners and stakeholders mentioned above have a vested interest in succeeding and therefore are committed to working towards common goals.

Evidence: The Office of Security Technology (OST) has ensured all Programmatic Documentation such as Statements of Work, Scopes of Work, Schedules, OST Program Plans, OST Acquisition Plan, etc., contain sections that move towards the long-term and/or annual goals. The program has also collaborated with a contactor by obtaining strategic input from internal and external stakeholders (i.e. airports and airlines) which produced the Baggage Screening Investment Study (BSIS) presented to the Aviation Security Advisory Committee (ASAC). The program works closely with the airport authorities and airlines to install safe in-line BHS. This removes systems from airport lobbies that hinder passenger flow and affect airport businesses. These in-line BHS can be easily scaled to match projected airline growth. The program also is replacing first generation in-line systems that do not reflect best design practices with more appropriate systems. The program works closely with the TSO's to reduce injuries while on the job, identify issues, and take remedial action when necessary.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: One of the most effective evaluation methods used continuously for the Passenger & Baggage Screening Technologies involves covert evaluations conducted by the GAO. The covert aspect allows for high quality results because of the uninhibited nature of examining effectiveness. As such, the covert evaluations are purposely conducted on asymmetrical intervals, but with a frequency that replenishes performance data when warranted. Due to the vivid exposure to any areas that require improvement or optimization within covert evaluations, the level of impact is substantial because remediation for deficiencies (if applicable) can occur with more expediency. While the covert evaluations are typically relative to processes and equipment effectiveness, there are other evaluations conducted that cover finance, capital investments, regulatory, and security-related matters. The pattern for these evaluations occurs on an annual basis with some occurring bi-annually and monthly as warranted by GAO and the DHS OIG. There is no conflict of interest between TSA and the entities that conduct evaluations (GAO and the DHS OIG). This provides for an effective, unbiased evaluation relationship which, in turn, allows for improved observations on program effectiveness.

Evidence: The evaluation plan for the Passenger & Baggage Screening Technologies program consists of covert GAO reports being conducted as-needed in order to replenish performance information and GAO/DHS OIG evaluations conducted for finance, capital investments, regulatory, and security. The criteria for selecting the GAO and the DHS Inspector General's Office is dictated by federal compliance rules and adhered to by TSA.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Passenger & baggage screening technology is uniquely identified in the TSA Budget requests and performance goals and targets are tied to the funding level via a performance based budget, which explicitly ties program accomplishments to funding. For example, the Percentage of Time Passenger Screening Technology is Operational and Percentage of Time Baggage Screening Technology is Operational annual measures are used to determine how effective the program's equipment maintenance dollars are being expended on an annual basis by providing an indication of whether anticipated uptime requirements for a given fiscal year are being met. In addition, the Percentage Increase in the Time Passenger Screening Equipment is Operational and Percentage Increase in the Time Baggage Screening Equipment is Operational long-term measures indicate whether the program has successfully made incremental improvements in equipment uptime from year to year. These measures collectively enable the program to monitor how well it is fulfilling its purpose in the area of equipment life-cycle management. All resources for the program are identified within its budget request, which ultimately becomes its Spend Plan. Both of which are thoroughly reviewed and approved by TSA, DHS, OMB and Congress. The Spend Plan is transparent and ties to the Department's Future Year Homeland Security Program (FYHSP), which is a system for monitoring Department wide performance and budget, including the full costs of the program, milestones and performance metrics. These costs, however, only represent the program's direct costs, such as salaries, benefits, travel, supplies, contractual services, and equipment. Indirect costs, such as utilities, fuel, rent, janitorial services, security services, human resources support, acquisition support, IT support, and financial support are currently not tracked on a program by program basis within TSA. Instead these costs are captured within the agency's Transportation Security Support appropriation, which covers these expenses for the entire agency nationwide. The capital assets acquired by the Passenger and Baggage Screening Technology Program contribute to the enhanced effectiveness and efficiency of the Department's overall Aviation Security program through improved methods of detection, increased capacity, and reduced human resource requirements for the equipment operation. These improvements are measured through several efficiency measures, such as the Cost Per Passenger Screened, the Cost Per Bag Screened, the Ratio of Passengers Screened to Passenger Screener Full Time Equivalents (FTE) Expended, and the Ratio of Bags Screened to Baggage Screener FTE Expended.

Evidence: All passenger & baggage screening technology is differentiated in the budget justifications and the justifications are organized in a performance-based structure.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program has taken the following steps to correct its strategic planning deficiencies: Strategic Requirement: The TSA CTO requires each program to develop a strategic plan. While the TSA strategic goals coincide with those of DHS, they are high-level concepts rather than measurable objectives. The MNS provides a detailed review of those goals and the program mission. Strategic Alignment: Specifically, the strategic plan offers the program's approach and solution for meeting near-term technology deployment objectives in consonance with the five-year, strategic passenger security goals. Short-term procurement objectives are reflected in the AP. Near-term performance objectives are identified in the APB. In essence, the strategic plan merges short term acquisition objectives to the strategic goals defined in the overarching approach.

Evidence: ?? APB ?? MNS ?? AP ?? Strategic Plan (2005 - 2008) ?? Baggage Screening Investment Study (BSIS) (2006) The Strategic Planning Framework (DHS Internal Version) was reissued on December 19, 2005 and it reflected modeling studies done in 2004-2005. The framework was intended for TSA and DHS only; and, as a result, it did not go to Congress. However, this was the first strategic plan for the program. The Report to Congress: "Strategic Planning Framework for the Electronic Baggage Screening Program" was submitted to Congress on February 8, 2006 and contains the same information as the DHS Internal Version, but was condensed for distribution to Congress. The data reflects modeling studies done in 2004-2005. The Working Group Report: "Baggage Screening Investment Study (BSIS)" was submitted to ASAC on August 9, 2006. The date it was submitted to Congress is unknown. This report was developed by the BSIS working group and is not an official TSA report. However, it contains the latest "official" modeling runs. It is unclear whether the full report was submitted to Congress or just the synopsis, which is available on TSA's website at http://www.tsa.gov/research/asac/index.shtm. The latest Strategic Plan was recently completed and is in the final stages of review by TSA OST. After the review has been finalized, it will be submitted to TSA, then subsequently submitted to DHS, OMB and then to Congress.

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: The program has performed several alternative analyses in 2005, 2006 and 2007 which are captured in three versions of the strategic plan and within the BSIS. All of these reports have been delivered to TSA, DHS, OMB and Congress except the 2008 Strategic Plan which is still under review in TSA OST. Each organization and Congress has provided input into the strategic plan which has altered future strategic planning. Accordingly, the APM and IDM are modified to reach the optimal system state as mentioned in the Strategic Plan on an annual basis as a direct result from the analysis performed on the viable alternatives. The practical solution has been the current approach developing a selection process that measures technical performance, acquisition and operating cost, throughput, and footprint. The acquisition process is designed to optimize the subsequent selection.

Evidence: ?? The Strategic Plan ?? The Baggage Screening Investment Study (BSIS) ?? The PMP ?? Program artifacts

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The agency does collect performance information for use in managing and improving performance on a regular basis. One example of this is the reliability, maintainability, availability (RMA) data kept on the passenger screening systems across the country. This information provides OST management with the means to monitor how the original equipment manufacturers and the maintenance vendors are performing on a monthly basis. It also enables the agency to halt further procurements should a system not meet contractual standards. In addition, it has enabled the maintenance department to make adjustments to their workforce and warehouse management processes in order to provide the agency with faster response times, lower downtime of equipment, and higher levels of operational availability. Other examples of data that are collected to manage and improve performance include, False Alarm data, which are used to adjust performance parameters in OEM contracts and the Acquisition Program Baseline (APB), as well as data from the agency's Performance Management Information System (PMIS), which are used to calculate the program's cost per passenger and bag screened efficiency measures.

Evidence: Monthy RMA reports from maintenance vendors, monthly program management reviews with vendors. Maintenance and False Alarm Data

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and program partners are held accountable. For example, the Office of Security Technology's Director is issued a Senior Executive Service (SES) performance-based work plan on an annual basis, which explicitly defines performance goals and objectives for the fiscal year. The Director is then held accountable for the actual results achieved upon receiving an annual rating. Program managers are also held accountable through acquisition directives, which assign individual accountability for cost, schedule, and performance. Furthermore, they are certified per DAIWA Level III and Project Management Institute (PMI) credentials. In addition, key program partners such as maintenance contractors, support contractors, general contractors, etc. are responsible for achieving results in accordance with TSA performance goals. This is achieved via contractual requirements, which are stated in performance terms. Furthermore, the program is developing improved performance standards in a new OEM contract solicitation for 2008 and beyond. In the past, the DHS Inspector General has found that TSA did not hold its primary equipment service provider accountable for performance. However, TSA responded by implementing measures, which hold vendors accountable for outstanding issues such as higher availability and reliability. Therefore, any equipment not meeting these standards will not be accepted into qualified product status until the vendors can prove they meet these requirements and can maintain such levels throughout the service life.

Evidence: Terms of primary support contracts. Program reviews by COTRs. The Director's Work Plan. Current contract documentation such as statements of work, scopes of work, and schedules include performance targets which tie to program goals. OEMs will be required to have earned value management systems for 2008 awarded contracts and future awarded contracts. For 2008, the program is in the process of changing the maintenance metric from fleet-wide to airport-specific, which allows for better monitoring of maintenance tracking and mean down time tracking. 2009 plans include: consolidating all rigging and installation into one contract; competing equipment purchases airport-by-airport vs. replacing equipment based on legacy status; and, implementing performance-based contracting and surveillance plans that will allow the program to perform random sampling of its performance, process, and procedures.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: The Passenger and Baggage Screening Technology Program's funds are obligated annually, however, the timeliness of the obligation can vary based on external factors. Of the $168M available for Passenger Screening Technology in the Fiscal Year (FY) 2007 Appropriations Act and prior year's appropriations, 98% was obligated and/or outlayed. Moreover, of the $529.4M available for Electronic Baggage Screening Technology in FY 2007 Appropriations Act and prior year's appropriations, 97% was obligated and/or outlayed. These funds are obligated based upon an established schedule. For example, the FY 2008 Omnibus Bill required the submission of an expenditure plan for the Passenger and Baggage Technology funds. These schedules are designed to fully meet the resource needs of the program, but the overall execution is based upon the approval of the spend plan. Once these plans have been fully approved by the Congress, contracts are executed. This layer of approval causes untimely and unforeseen delays. Commitments and expenditures that result from these contracts are all tracked by the TSA CFO, the Business Management Office and the program office on a weekly basis throughout the fiscal year. Each office reconciles their databases to ensure the reporting of these financial transactions is accurate. In addition, after the award of contracts, procedures exist for reporting actual expenditures into the Federal Procurement Data System - Next Generation (FPD-NGS) in a timely manner to confirm that the funds are being spent for their intended purposes.

Evidence: The agency maintains a Status of Funds and periodic program reviews are conducted to include a detailed expenditure report. Internally, a weekly internal log is kept, while FPD-NGS is used externally.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The Passenger & Baggage Screening Technologies Program has adopted appropriate procedures to ensure the efficiency of its operations and routinely examines its business practices to ensure cost effectiveness and to identify opportunities for savings. These include: (1) establishing management goals that are focused on minimizing the time to bring new systems to maturity (2) publishing the Acquisition Plan (AP) and defining contractual needs and cost parameters in the Acquisition Program Baseline (APB) (3) monitoring the cost per passenger and bag screened on a regular basis(4) using both security requirements and costs to make competitive sourcing decisions and (5) coordinating across programs to gain efficiencies on contracts and pricing. The program's Strategic Plan outlines the sourcing/cost comparisons that have been conducted throughout the year to determine the most cost effective manner in which to achieve an optimal system state. Efficiency is primarily measured as cost per passenger screened and the cost per bag screened.

Evidence: Recent contract, AP, APB, Quarterly Metric Submissions, Efficiency measure for baggage screening, Strategic Plan, Integrated Deployment Model (IDM)

Does the program collaborate and coordinate effectively with related programs?

Explanation: In order to meet National and DHS security requirements, both the Passenger and Baggage Screening Technology Programs must collaborate in order to secure multiple aspects of aviation security. By sharing resources and working with similar vendors and technologies, efficiencies are gained. They must also collaborate with external stakeholders and partners, which include but are not limited to: commercial airports, the Office of Security Operations (OSO), the Security Technology Integrated Program (STIP), and the Aviation Security Advisory Committee (ASAC) so that they can examine the full spectrum of aviation security. Furthermore, these partnerships enable the program to allocate equipment and to make decisions about checkpoint configurations, as well as enable partners to make decisions about manpower and training requirements. Specific examples of these collaborative efforts include the 2006 Baggage Screening Investment Study (BSIS) report which was entirely developed by external stakeholders and the Planning Guidelines and Design Standards for Checked Baggage Inspection Systems, which was published on October 10, 2007 and provides a consolidated source for best practices and standards that must be followed to ensure optimal system performance.

Evidence: August 9, 2006 Working Group Report: Baggage Screening Investment Study (BSIS) October 10, 2007 Planning Guidelines and Design Standards for Checked Baggage Inspection Systems Interagency agreements, meeting minutes, and Project Charters

Does the program use strong financial management practices?

Explanation: TSA has significantly improved its financial management practices through a number of initiatives in recent years. TSA has delegated funds certification directly to the Assistant Administrator of each organization for increased accountability; fund allocations are provided at the management unit levels versus (vs) the Program/Project/Activity level; eliminated old accounting codes to decrease likelihood of miss-coding; institutionalized funds control to provide warnings of low account balances; reduced carryover funds by 50% in 2006 vs. 2007; and TSA has issued annual allocations for fiscal year (FY) 2008 in August 2007 -- four months earlier than the previous FY. The program itself maintains internal accounting logs to record transactions against the Checkpoint Spend Plan for day-to-day activities. Additionally, it collaborates with the acquisitions team to analyze contracting methods to ensure the government is receiving the best value on its procurements. Other controls include segregation of duties (separate personnel with authority to authorize a transaction, process the transaction, and review the transaction); appropriate documentation and role-based access to such documentation; and quarterly reviews of the budget to ensure funds are being utilized for approved spend plan items.

Evidence: Management controls, Status of Funds, Finance and Procurement Desktop, Procurement Request logs, Expenditure Plan, AP, APB and quarterly program reviews

Has the program taken meaningful steps to address its management deficiencies?

Explanation: TSA has developed and implemented a Mission Action Plan to correct material weakness, and anticipates that remediation will be reflected in the FY 2008 financial audit. As part of TSA's effort to properly account for funds and improve financial record accuracy, programs are conducting quarterly reviews and validation of past obligations. In accordance with the 2006 appropriations language, the program developed a detailed Strategic Capital Plan, which will guide management and equipment deployment for the next 5-10 years. It also has a firm-fixed price equipment maintenance contract in place and is reviewing the current maintenance strategy to determine if effective cost improvements are feasible.

Evidence: Program charters, designation certificates, contracts. EBSP Strategic Plan (Versions from 2005, 2006) EBSP Baggage Screening Investment Study (BSIS) (2006) EBSP Strategic Plan (Draft 2008) EBSP Integrated Deployment Model (IDM) EBSP Airport Prioritization Model (APM) EBSP Acquisition Program Baseline (APB)

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: The program defines deliverables, performance characteristics, credible cost and schedule goals in the approved Acquisition Plan (AP) and Acquisition Program Baseline (APB) that it has been maintaining and following since FY05. The AP provides the program with clearly defined milestones and goals, as well as a foundation from which it may guide vendors and equipment manufacturers to deliver the products necessary for the agency to be successful. Within the AP are clearly defined milestones and goals. The APB defines specific cost, capabilities and performance characteristics required for the successful introduction of these emerging technologies. The program also uses statements of work and operational requirements documents to guide vendors and show them the deliverables, schedules, and performance characteristics they will need to deliver within the products and services. In order to minimize costs the program primarily employs contracts with either fixed price or time and material with not to exceed clauses (effectively making them fixed price) to get the services needed within cost and schedule goals. Furthermore, the program has developed a repeatable OMB 300, APB and AP since 2006. These documents are updated regularly and clearly define the performance, schedule and cost parameters of the program. They are also used for program management and for gathering input from internal and external stakeholders. Additionally, to ensure an integrated approach, the technical screening baggage community is employed as part of this input process. The program also uses statements of work and operational requirements documents to guide vendors and show them the deliverables, schedules, and performance characteristics they will need to deliver within the products and services. Furthermore, the program has developed a repeatable OMB 300, APB and AP since 2006. These documents are updated regularly and clearly define the performance, schedule and cost parameters of the program. They are also used for program management and for gathering input from internal and external stakeholders. Additionally, to ensure an integrated approach, the technical screening baggage community is employed as part of this input process.

Evidence: Acquisition Plan, Acquisition Program Baseline (APB), OMB 300s, Future Year Homeland Security Program (FYHSP)

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has demonstrated progress against its long-term performance goals. The Percentage Increase in the Time that Passenger Screening Equipment is Operational increased by .75% between the baseline year of FY05 and FY06. In FY07, however, the increase over the FY05 baseline was a modest .12%. While the FY07 actual fell short of the .75% target, it was still an improvement over the baseline year. Furthermore, it provides an indication of how ambitious the established targets are. The Percentage Increase in the Time that Baggage Screening Equipment is Operational Increased from 1.04% between the baseline year of FY05 and FY06. In FY07 this percentage change jumped to 2.08%, which was well above the established target for a percentage increase of 1.10%. Due to this substantial increase, this measure's out year targets will be re-evaluated at the conclusion of FY08.

Evidence: PART submittals, OMB 300s, PART Improvement Plan, Strategic Plan, Integrated Deployment Model (IDM), Airport Prioritization Model (APM), Acquisition Program Baseline (APB)

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program has demonstrated achievement of its annual performance measures. For example, the Percentage of Time Passenger Screening Equipment is Operational measure exceeded its original target of 98% in both 2006 and 2007. Furthermore, the program came within .1% of its targets for the Percentage of Time Baggage Screening Equipment is Operational during FY07. In addition, the Ratio of Passenger Screener to Passenger Screener Full Time Equivalent (FTE) Expended and the Ratio of Bags Screened to Baggage Screener FTE Expended, exceeded their respective FY07 targets of 27,650 and 39,320 by 64 passengers and 1,568 bags. The Cost Per Passenger Screened measure also exceeded its target of $0.13 and came in at $0.09 per passenger during FY07. The Cost Per Bag Screened Measure, however, was re-baselined in FY07.

Evidence: Strategic Plan, Acquisition Program Baseline (APB), Quarterly/Annual Management Review metric reports The ratio of bags screened to baggage Screener FTE expended has steadily increased each year. From FY 2005 to FY 2006, the ratio increased by 12% and from FY 2006 to FY 2007, it increased by 10.2%.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has demonstrated improved efficiency and cost effectiveness through several measures. These include the Cost Per Passenger Screened measure, which was only $0.09 during FY07, exceeding its target of $0.13. In addition, the Ratio of Passenger Screener to Passenger Screener Full Time Equivalent (FTE) Expended, which is an indicator of labor requirements, was 27,714 during FY07, exceeding its target by 64 passengers. These indicators show that the cost of screening each passenger remains to be pennies on a dollar even as the capability to screen more passengers with fewer Transportation Security Officers has increased. These efficiencies have been gained through the deployment of advanced passenger screening technologies and will continue to improve as the technology improves and as it becomes more widely available. There are also efficiency measures for baggage screening technology. The Cost Per Bag Screened measure was re-baselined in FY07 and currently stands at $3.27 per bag. This baseline, however, was used to establish a new target if $3.35 for FY08. Although performance on this particular measure is still pending the program has shown demonstrative improvement in the Ratio of Bags Screened to Baggage Screener FTE Expended. During FY07, the ratio was 40,888 bags per baggage screener FTE. This was not only 1,568 more bags than originally targeted, but was also an improvement of 3,791 bags over FY06. This improvement is largely attributed to the continuous deployment of in-line baggage handling systems, which are highly automated systems with conveyor belts that automatically route bags to their intended flights and that provide the TSOs with the ability to examine and resolve most detected threats remotely with a computer thereby requiring fewer bags to be hand searched. These efficiencies result in increased baggage handling capacity, reduced personnel injuries, and a reduced need for screeners to operate the equipment.

Evidence: Contracts and Acquisition Plans Efficiency measure for baggage screening, Integrated Deployment Model (IDM) Thus far, the efficiency results have shown a slight reduction in the cost per screened bag from the 2007 calendar year.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: TSA performs a unique function of coordinating the security of a complex, dynamic heterogeneous transportation network comprised of U.S. federal, state, local, quasi-governmental, tribal and private entities. Because only the Surface Transportation Security Program develops national security policies and programs for the identification, prioritization and protection of critical infrastructure and key resources across all surface modes in the Transportation Sector, the performance of this program cannot be compared to other domestic government or private programs. No comparative evaluations to other international programs have been conducted or planned. However, through participation in various international forums TSA has learned that although international agencies employ similar types of security enhancement activities, TSA's surface transportation security effort advances a much more integrated approach across all surface transportation modes, through expanded operational activities (random, unpredictable deterrence), creating stakeholder partnerships, building security force multipliers (training, exercises, public awareness activities), and developing security technologies.

Evidence: See the Aviation and Transportation Security Act (ATSA), Public Law 107-71 (Sections 101 and 137).

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations have been conducted by GAO and OIG. These evaluations have shown that the program is achieving results. These reports indicate that programs is performing well and, according to the OIG, the "security layers??are effective and formidable". The GAO reports state that the program "has made progress since 9/11 to reduce aviation related vulnerabilities and enhance the layers of defense directly exploited by the terrorist hijackers". The program is standing up an independent Integration Facility where various types of equipment and systems can be independently and routinely tested to determine improvements in technology, system integration, system designs and effectiveness (Reliability, Maintainability and Availability (RM&A) without having to impact operations in the airports. In the past, TSA could not perform testing in an operational environment because this would cause delay and disruption at the airport. This facility will be operational in April 2008.

Evidence: The DHS IG has found that TSA did not in the past hold its primary equipment service provider accountable for performance. In response, TSA improved its future contracts in this area in the 2005 - 2007 OEM contracts. The program is again developing improved performance standards in a new OEM contract solicitation for 2008 and beyond. GAO-07-375 , OID-08-25 February 2008

Were program goals achieved within budgeted costs and established schedules?

Explanation: The program has always kept obligation within the appropriations available and the current performance measures of operational availability has been met over the last three years. Furthermore, the program has met the security goals of enhancing legacy systems, expanding scope of threat detection, and of procuring and deploying emerging technology focused on explosives detection. While goals were achieved within budget, the cost and schedule targets change several times during any given fiscal year primarily based on late approval of the Spend Plan. In some years, the Spend Plan is not approved until late 2nd Qtr which does not allow the program to complete negotiations with various airports (JFK, LGA, LAX, etc). These airports require over 8 months of negotiations to establish a Letter Of Intent (LOI).

Evidence: The Electronic Baggage Screening Program program area has never gone over budget since its inception and has met each of its annual goals. This program received a supplemental amount of funding that is considered no year money which will allow the program to negotiate with the airports listed above to establish an LOI.