Hazardous Material Transportation Safety and Security Field Operational Test Final Detailed Test Plans

EXECUTIVE SUMMARY

The tragic events of September 11, 2001, and the more recent events of war with Iraq during the Spring of 2003, resulted in a significant heightened level of concern from federal government officials and transportation industry members regarding the secure transport of hazardous materials. These security issues focus on HazMat shipments as potential targets for terrorists. HazMat shipments through intermodal connectors, modes, and facilities are all prospective targets for domestic acts of terrorism, and pose a much greater concern to public safety than most other shipment types. HazMat shipments, especially fuels and chemicals, present an attractive target for terrorists due to the multiple points of vulnerability. These vulnerabilities exist at shipper, motor carrier, and shipment recipient facilities and shipment movement en route throughout the nation’s roadway infrastructure.

The Federal Motor Carrier Safety Administration (FMCSA), working in close cooperation with the Transportation Security Administration (TSA), has proactively addressed public and private sector HazMat security concerns by identifying potential security risks related to HazMat transportation and proposing solutions to minimize those risks. FMCSA embarked on a program to improve HazMat security and safety by using regulatory measures, security assessments, and outreach efforts.

Science Applications International Corporation (SAIC) is leading the Evaluation Team effort with assistance from Cambridge Systematics, Inc. (CSI). The Evaluation Team will appraise the impact of selected technology solutions regarding the security, safety, and operational efficiency of HazMat movements from shipper to en-route transport to final delivery. Particular emphasis will be placed on revealing levels of vulnerability within the HazMat movement chain by conducting a risk assessment. The Evaluation Team will coordinate activities with the Operational Test Team, led by Battelle (with individual task and deployment assistance from Total Security Services International, Inc. [TSSI] and Qualcomm), to obtain quantitative and qualitative test-generated data and information. This data and information will be used to develop a benefit-cost assessment for test component and integrated systems for improving HazMat security, safety, and operational efficiency.

The objective of this Hazardous Material (HazMat) Transportation Safety and Security Field Operational Test (FOT) Final Detailed Test Plans evaluation is to measure the impact of technology solutions on the safety, security, and operational efficiency of HazMat movements from shipper to en-route transport to final delivery. During the FOT, a suite of technologies will be tested across nine distinct operational/technology scenarios. These are comprised of four base scenarios, of which each represent a key segment of the HazMat industry. Within each segment, there are at least two participating companies using multiple technology suites to test the broadest combination of technologies in the multiple operational environments. Tables ES-1 through ES-4 provide an overview of the salient features of each scenario, summarizing the operational characteristics, the evaluation approach, the technologies tested, and the breakout of the sub-scenarios.

Throughout the FOT, data will be collected for the following elements:

· Baseline carrier operations

· FOT technology event capture

· Participant perceptions of technology efficacy and practicality

· Motor carrier industry and vendor community alternatives to the FOT test technologies

· Results of staged events

These data elements are designed to test the combinations of FOT technologies under conditions simulating attacks on HazMat loads.

This Detailed Test Plans document describes the evaluation in hierarchical order, from methodological overview to greater levels of detail of data elements and evaluation activities, in three related report sections:

· The evaluation methodology (Section 2) – which describes the research approach, measurements to be made, types of data that will be collected to support the analyses, and the analytical framework for conducting the evaluation assessments. This section enables the definition of the evaluation assessments, neutral of the complex association of technologies to individual test scenarios, in order to present concise descriptions of analytical approach.

The linkage of the elements presented Section 2 to the two subsequent sections and to the Deployment Team’s project documentation is based on the individual test technologies and their previously defined relationships to functional requirements, vulnerabilities, and deployment scenarios.

· Component technology functions and test data streams (Section 3) – provides details on the key functional and technical features for each of the component technologies at the individual technology level. Along with descriptions of the component technologies, there are specific examples of potential data collection elements for each applicable component technology. Section 3 augments the higher level functional and data descriptions presented in Section 2 (evaluation methodology), albeit on a single technology basis. The importance of this section is that it creates a logical linkage from evaluation method to scenario-specific evaluation activities, described below.

Table ES-1. Scenario 1 Basic Configuration Parameters

Configuration Parameters	Description
Operational Characteristics	The Bulk Fuel Delivery scenario will test the application of several technologies to improve the security, safety, and operational efficiency of delivering truckload quantities of fuel to retail locations. This scenario will involve short haul fuel delivery vehicles transporting Class 3 (Flammable Liquids).
Evaluation Approach	· Safety and Security Evaluation Approach – Ability of technology systems to enhance detection and response to illicit activities through vehicle tracking, mobile communications, driver authentication, emergency alerts and the ability to disable a vehicle. · The assessment is guided by FOT technical data, user perceptions, and staged tests as directed and reviewed by an Expert HazMat Security Panel and Delphi Group. · Operational Efficiency Evaluation Approach – Ability of technology systems to enhance motor carrier operational performance through vehicle tracking and mobile communications.
Test Technologies	· Wireless Satellite Communications (with GPS). · Wireless Terrestrial Communications. · Driver Authentication with Global Login. · Intelligent Onboard Computers (OBC) with Remote Vehicle Disabling Dispatcher or parameter set. · Panic Button in dash. · Panic Button Wireless/Remote with Remote Vehicle Disabling and Remote Emergency Notification.
Sub-Scenarios	· 1A Dupre/Bulk Fuel Delivery · 1B Cox Petroleum/Bulk Fuel Delivery/Configuration 1 · 1B Cox Petroleum/Bulk Fuel Delivery/Configuration 2 · 1B Cox Petroleum/Bulk Fuel Delivery/Configuration 3 · 1B Cox Petroleum/Bulk Fuel Delivery/Configuration 4

Table ES-2. Scenario 2 Basic Configuration Parameters

Configuration Parameters	Description
Operational Characteristics	The LTL High Hazard scenario will test the application of several technologies to improve the security, safety, and operational efficiency of delivering less than truckload quantities of liquid chemicals. This scenario will involve transporting LTL High Hazard materials. A portion of the FOT will monitor a lower cost (terrestrial hardware) technology installed on a national LTL fleet with a very high degree of integration and efficiencies.
Evaluation Approach	· Safety and Security Evaluation Approach – Ability of technology systems to enhance detection and response to illicit activities through vehicle tracking, mobile communications, driver authentication, emergency alerts, and the ability to disable a vehicle. · The assessment is guided by FOT technical data, user perceptions, and staged tests as directed and reviewed by an Expert HazMat Security Panel and Delphi Group. · Operational Efficiency Evaluation Approach – Ability of technology systems to enhance motor carrier operational performance through vehicle tracking and mobile communications.
Test Technologies	· Wireless Satellite Communications (with GPS). · Wireless Terrestrial Communications. · Driver Authentication with Global Login. · Intelligent Onboard Computers (OBC) with Remote Vehicle Disabling Dispatcher or parameter set. · Panic Button in dash.
Sub-Scenarios	· 2A Distribution Technologies/LTL High Hazard · 2B Roadway/LTL High Hazard

Table ES-3. Scenario 3 Basic Configuration Parameters

Configuration Parameters	Description
Operational Characteristics	The Bulk Chemical scenario will test the application of several technologies to improve the security, safety, and operational efficiency of delivering bulk truckload quantities of chemicals. This scenario will involve chemical delivery vehicles transporting chemical, plastic, industrial, and agricultural products.
Evaluation Approach	· Safety and Security Evaluation Approach – Ability of technology systems to enhance detection and response to illicit activities through vehicle tracking, mobile communications, driver authentication, emergency alerts, ability to disable a vehicle, and to authenticate and document load information and chain of custody. · The assessment is guided by FOT technical data, user perceptions, and staged tests as directed and reviewed by an Expert HazMat Security Panel and Delphi Group. · Operational Efficiency Evaluation Approach – Ability of technology systems to enhance motor carrier operational performance through vehicle tracking, mobile communications, and real-time access to shipment information and status.
Test Technologies	· Wireless Satellite Communications (with GPS). · Driver Authentication with Biometrics and Smart Cards. · Electronic Supply Chain Manifest (ESCM). · Panic Button in dash. · Panic Button Wireless/remote with Remote Vehicle Disabling and Remote Emergency Notification.
Sub-Scenarios	· 3A Transport Services · 3B Quality Distribution · 3C Roeder Cartage

Table ES-4. Scenario 4 Basic Configuration Parameters

Configuration Parameters	Description
Operational Characteristics	The Truckload Explosives scenario will test the application of several technologies to improve the security, safety, and operational efficiency of delivering truckload shipments carrying explosive materials. This scenario will involve Truckload Explosive delivery vehicles, delivering explosive blasting type A, Class 1.1 - 1.6 explosives. One carrier in this scenario performs many multiple load deliveries often with six to eight specific loads per truck.
Evaluation Approach	· Safety and Security Evaluation Approach – Ability of technology systems to enhance detection and response to illicit activities through vehicle tracking (both tractor and trailer-with alerts of unauthorized untethering), mobile communications, driver authentication, emergency alerts, ability to disable a vehicle, provide tamper-proof securement of cargo, and to authenticate and document load information and chain of custody. · The assessment is guided by FOT technical data, user perceptions, and staged tests as directed and reviewed by an Expert HazMat Security Panel and Delphi Group. · Operational Efficiency Evaluation Approach – Ability of technology systems to enhance motor carrier operational performance through vehicle tracking (with out of route alerts), mobile communications, and real-time access to shipment information and status.
Test Technologies	· Wireless Satellite Communications (with GPS). · Driver Authentication with Biometrics and Smart Cards. · Electronic Supply Chain Manifest (ESCM). · Intelligent Onboard Computers (OBC) with Remote Vehicle Disabling Dispatcher or parameter set. · Intelligent Onboard Computers (OBC) with remote Trailer Door Lock · Panic Button in dash. · Panic Button Wireless/remote with Remote Vehicle Disabling and Remote Emergency Notification. · Electronic Cargo Seals. · Tethered and Untethered Trailer Tracking. · Routing and Geo-Fence Mapping Software.
Sub-Scenarios	· 4A R&R Trucking/Truckload Explosives/Configuration 1 · 4A R&R Trucking/Truckload Explosives/Configuration 2 · 4A R&R Trucking/Truckload Explosives/Configuration 3 · 4B Dyno Transportation/Truckload Explosives/ Configuration 1 · 4B Dyno Transportation/Truckload Explosives/ Configuration 2

· Scenario-specific evaluation activities (Section 4) – provides a detailed description of the specific evaluation activities planned for each sub-scenario. The component technology functions and test data streams section describes each planned deployment component, while this section groups them into the defined scenarios. This information summarizes the technology deployments for each of the sub-scenarios and identifies specific data collection opportunities and mechanisms and provides a brief overview of the scenario characteristics. Although the data collection activities are defined by technology, the anticipated benefits are tied to specific scenarios, creating a systems approach to the data collection and assessment of the FOT. This creates the framework for a systems approach to the evaluation.

Overall Evaluation Framework

The purpose of this FOT is to test methods for leveraging technology and operations to improve HazMat transport security, safety, and operational efficiency. As such, technologies will be demonstrated which can decrease the existing vulnerabilities of HazMat shipments to terrorist activities. The ability of technology, combined with changes in shipper/carrier operations, to achieve this goal will be tested across four distinct HazMat operational scenarios, as defined in the Concept of Operations. In addition to the deployment technology components and systems impact on security, the evaluation will also focus on safety and operational efficiency impacts. These areas will be critical to the success of the FOT as safety and operational efficiency benefits are more directly relevant to a typical HazMat motor carrier’s daily operations. While this does not dismiss the security concern that faces HazMat motor carriers, it does recognize the reality that the number of domestic terrorist attacks on HazMat shipments is so infrequent that carriers naturally focus more on accident reduction and improving operational efficiency. The approach to this evaluation will encompass assessing technology solutions aimed at improving security, safety, and operational efficiency throughout the HazMat distribution chain.

Figure ES-1 illustrates the framework of the overall evaluation, which begins with the Concept of Operations that was defined by the Operational Test Team. The Concept of Operations document provides the specific technology deployments for each of the four FOT scenarios. Based largely on this document, the Evaluation Team identified the scenario-specific operational parameters to be examined, which dictate the Evaluation Team’s data and information collection efforts. The specific technologies and level of deployments are defined for each designated FOT participant. Based upon the actual deployments, specific data that becomes available can be identified. As shown in Figure ES-1, data will fall into two basic categories: qualitative versus quantitative, and field versus staged. These data elements will be used to measure the effectiveness of each FOT scenario (impact of technologies, participant acceptance, level of use, etc.).

Figure ES-1. Evaluation Framework

The evaluation will focus on the ability of technologies and procedures to reduce HazMat vulnerabilities; identify potential benefits and costs of technology solutions; determine the deployment potential for technologies; and identify the institutional factors that need to be addressed to realize full deployment potential. These will be examined within the framework of three key evaluation areas: Safety and Security, Operational Efficiencies; and Institutional Challenges, and are described as follows.

Safety and Security

The Safety and Security assessment is designed to assess the test technologies and technology suites, which are composed of multiple technologies; and the ability to enhance motor carrier, shipper/consignee, and enforcement detection and response to terrorist actions against HazMat shipments. The individual test technologies, technology suites, and deployment scenarios developed for this FOT were based on the Deployment Team’s identification and prioritization of HazMat shipment vulnerabilities. These vulnerabilities focus on the illicit commandeering of HazMat through pilferage and accumulation of poorly managed shipments or attacks directed against drivers/vehicles/loads en route to obtain and use shipments for attacks against individuals and property. The Safety and Security assessment will determine the ranges of costs associated with terrorist attack countermeasures (technology costs) proportional to ranges of consequence avoidance (benefits).

Using the preliminary threat, risk, and consequence analyses developed by Battelle and TSSI as a starting point, SAIC will validate the initial findings and collate the expert opinions of counter-terrorism professionals, law enforcement, and HazMat shipping stakeholders to address the goals of the risk assessment (i.e., what are the greatest risks, do the solutions being tested effectively address the risks and to what extent, and what more needs to be done to secure the nation’s hazmat shipments?).. A core advisory group (Expert Panel) will shape the process, with significant input provided via Delphi surveys administered to a larger working group of subject experts.

Expert Panel

The collation of expert opinion will be managed through the direction from a core advisory group consisting of 10 project-sponsored or volunteer experts in HazMat risk and loss prevention and intervention. This “Expert Panel” will be staffed by: members of the FOT deployment and evaluation teams; industry security experts from the American Trucking Associations (ATA); Chemical Manufacturers Association (CMA); representatives from the Commercial Vehicle Safety Alliance (CVSA); leading expert members of associations of fire and police chiefs; the U.S. Department of Transportation; Transportation Security Administration (TSA); and the U.S. Department of Environmental Protection.

The Expert Panel will provide guidance on research methodology, critical analytical assumptions, data collection instruments, and subsequent research findings. Additionally, the Expert Panel will be asked to contribute its knowledge to the design of staged exercises or test events for the FOT technologies in three operational scenarios. The staged events will be designed to exercise the full integration of the technology suites as well as the technology/human interface to effectively detect and respond to staged attacks on HazMat shipments. The panel recommendations for conducting the staged events will be coordinated with the FOT participants to ensure realistic representation of likely attack scenarios and intervention processes to assess the level of success in defeating the attacks.

The Delphi Group and Delphi Survey Effort

The Delphi Group will be comprised of a collection of 25 or more individuals, either familiar to the members of the core advisory group, and/or, previously identified through their affiliation with associations, conferences, or working groups (notably the FMCSA HazMat Working Group, recruited to support this effort) that focus on freight security issues.

The Delphi Group will reflect the composition of the Expert Panel in terms of expertise and knowledge of HazMat shipping issues, technology usage, and security and risk assessment. With input from the Expert Panel, three Delphi surveys will be administered with increasing levels pre-knowledge of the FOT findings. These surveys will be developed to establish or validate previously developed linkages between threats to load types/carrier, potential consequences, and countermeasure (technology and procedural) effectiveness to address potential attacks on or using HazMat shipments.

The information developed through these activities will be combined with technology performance from the FOT. Additional resources will include numerous secondary sources to assign ranges of benefits and costs of technology systems in the context of attack consequence avoidance.

Operational Efficiencies

The goal of this evaluation area is to assess the test technologies and technology suites ability to enhance motor carriers’ financial performance. If positive returns on investment can be demonstrated by applying the technologies in daily fleet management operations, coupled with the estimated safety and security benefits, then motor carriers would more likely to be willing to invest in the systems. As a result, the motor carriers would gain the safety and security benefits at little or no additional investment cost on their part.

Site visits with participating motor carriers led to the determination that regardless of whether systems are primarily paper-based or highly automated, at least one or more of key business metrics are tracked by the trucking managers to measure the performance of their businesses. Therefore, direct statistical comparison of baseline operating conditions to the FOT operation conditions will enable quantitative analyses to determine whether detectable and significant operational enhancements are achieved via the test suites.

Institutional Challenges

The Safety and Security and Operational Efficiency analyses will establish the relative effectiveness of the test technologies and technology suites. This analysis will also test the relative effectiveness test of commercial-off-the-shelf (COTS) technology systems (with similar functionality to the test technologies). This combined analysis will address critical vulnerabilities and enhance a motor carrier’s bottom line. Potentially constraining the expansion of the FOT concepts to the larger universe of HazMat trucking transportation will undoubtedly involve a number of institutional challenges that would need to be addressed to promote industry wide deployment of the technologies.

Key perceptions concerning technological efficacy; cost effectiveness; applicability to operations; processes and practices changes; trends in HazMat carriage; and direction of current and future security initiatives will be collected via interviews with FOT participants. These participants will include motor carriers, enforcement and response personnel, and technology deployers; industry stakeholders; government officials; and technology vendors before, during, and post FOT. The perceptions will allow the Evaluation Team to assess factors that may limit or promote the use of technology-enabled security enhancements. In the case of limitations, suggested countermeasures will be solicited from interviewees.

Evaluation Outputs

The analytical framework, supported through the collection and processing of detailed data elements combined with secondary data sources and expert opinion, described in the Detailed Test Plans, will enable the Evaluation Team to:

· Assess the ability of technology systems and technology-enhanced motor carrier and enforcement processes to reduce HazMat vulnerabilities and the potential consequences of terrorist attacks using HazMat shipments as weapons of mass destruction.

· Identify operational efficiency benefits by assessing the effectiveness of the operational test technologies to improve operational efficiency throughout the HazMat movement chain.

· Detail costs for individual test system components and integrated HazMat systems leading to a cost analysis that estimates the cost of deploying these technologies (both individual components and integrated systems) throughout the freight industry.

· Present a relevant benefit-cost analysis for individual test components and integrated test systems covering both industry return on investment (ROI) benefit-cost, and societal benefit-cost from reduced vulnerabilities to terrorism and improved safety.

· Establish ranges of safety, security, and operational benefits and costs associated with the technology systems by key segments of the HazMat trucking industry.

· Working with the Deployment Team, provide a comprehensive compendium of promising commercially available technology solutions across a wide spectrum of implementation costs.

· Develop a comprehensive understanding of the financial, procedural, and risk issues that need to be addressed to encourage adoption of the most promising technology solutions, and make recommendations for additional programmatic support of the goals of the FOT.

· Assess the potential and develop recommendations for widespread deployment in the HazMat industry for both component test technologies and integrated test systems.

Evaluation Continuous Process Improvement

While this document provides the third and final version of the Detailed Test Plans for the conduct of this evaluation, the Evaluation Team recognizes that the dynamic nature of this project will require improvements and updates to these plans. Therefore, for any significant updates or changes to these plans, the Evaluation Team will publish Technical Memorandum, which will provide the details of the updates or changes. It is expected that several of these “Evaluation Tech Memos” will be published during the remaining timeframe of this effort.

PDF Version

Federal Motor Carrier Safety Administration