Background

Project Overview. The Federal Motor Carrier Safety Administration (FMCSA) Office of Research and Analysis (R&A) conducted an industry meeting in Miami, Florida on February 9, 2006 to solicit industry input on two draft Statements of Work (SOW) for a motor carrier efficiency study as required in Section 5503 of the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users (SAFETEA-LU) of 2005. Eighteen representatives from trucking industry associations and consultants attended. The attendance list is attached.

Jeff Loftus, the Program Manager from FMCSA began the meeting by providing an overview of the Section 5503 requirement. The following is the text of that section:

SEC. 5503. MOTOR CARRIER EFFICIENCY STUDY.

Mr. Loftus explained that the government was looking for input on general concepts and requirements described in the draft SOWs.

Contracting Officer's Overview. Ruby Mixon, the Contracting Officer for this project, is one of two warranted contracting Officers with the FMCSA, an agency that is part of the U.S. Department of Transportation. Ms. Mixon provided the following Contracting Officer's Overview for this project:

The Acquisition Office consists of a Division Chief and approximately 15 Government and contracted support staff. The office acts as business advisors for customers so that their program needs are procured in a timely efficient manner. Currently, there are only two individuals who can legally obligate for the Government and only within the stated warrant limits. Ms. Mixon reviewed the ground rules for the meeting in order to comply with the guidelines of the Procurement Integrity Act. These ground rules protect the Government as well as industry: Government is protected by the ground rules by minimizing the possibility of litigation due to access to information to some participants and not all. Industry is protected by the ground rules because they allow industry to participate in industry capabilities meeting and not jeopardize their opportunities to submit proposals in the resulting procurement once published.

PROCUREMENT PROCESS – Ms. Mixon stated the procurement process was a continuous cycle of events. Depending on the approach, industry would be involved in several of the stages.

Need Analysis – Ms. Mixon stated the Government would determine that it had a need and then determine the best way to identify it and convey to industry the outcomes to be achieved. Sometimes industry would be asked to participate in assisting the Government in developing the SOW that identifies the outcomes such that it meets the Government requirements, was biddable, realistic, quantifiable and measurable. Ms. Mixon explained that this was the purpose of the February 9th meeting. The Government would identify in broad terms what they hoped to achieve - end results. She continued by stating that the attendees' participation in the meeting would assist the Government in gathering information that may be used in developing the final SOW.

Published Requirement – Ms. Mixon explained that the published requirement would be in the form of a solicitation that would outline the Government requirements and provide industry with the instructions for submitting proposals. It would describe the acquisition strategy identifying award type, performance period, factors that would be evaluated, how the awardee would be selected, and the date/time for response.

Proposal Receipt – She continued by explaining that proposals received from industry that were timely would be sent to the Government evaluation team for written evaluation. That process may or may not involve a discussion period with bidders.

Award – Ms. Mixon said that once the evaluation was completed, a decision would be rendered by the Source Selection Official, who would normally be the Contracting Officer, and notification would be made to all bidders who were timely and still under consideration for award.

Performance – Ms. Mixon continued by stating the selected contractor would perform in accordance with the requirements outlined in the SOW.

Monitor and evaluate quality – She said the Government surveillance of the contractor's performance would ensure that the requirements would be met in accordance with the performance objective and measures outlined in the SOW. The Government would document the quality of the contractor's performance. This would be communicated in monthly or quarterly reviews or as determined in Contractor’s Discrepancy Reports.

Review Outcomes – Ms. Mixon explained the Government would review the outcomes generated from the contract and make a determination that either the overall objective had been achieved or more work would be required which would start the process over again.

Evaluating Factors – Ms. Mixon explained it was premature to discuss factors at the meeting.

Pre-proposal Conference – She continued by stating that the meeting was not a Pre-proposal Conference. She said a solicitation has not been developed at this point. She pointed out that industry's input at the meeting was crucial in assisting the Government in developing the SOW that will be included in the solicitation when it is released. She explained a notice would be published on the Federal Business Opportunity web page to provide the projected date of release.

Published Report – Ms. Mixon explained a report of the results from this session would be made available for publication to industry with a time frame (cut-off time) for any additional comments. This would be done to ensure that everyone has a fair opportunity to compete and would be privy to the same information. She asked the participants not to hold side-bar discussions with any Government representative in attendance.

Summary of discussions in response to Government's questions

Mr. Loftus reported that each of the participants had been sent, in advance of the meeting, a copy of nine questions the Government wanted the attendees to answer. The nine questions were:

Question 1: Are these the right tasks? Any missing?

Generally, the participants felt the tasks were the right ones and that no tasks were missing. There were a few questions seeking to clarify the tasks. The following comments and questions are included to represent the consideration of this question:

One participant commented that instead of looking at corridors, the Government should look at supply chains because they are easier to define.

Another participant suggested the study should be broad and holistic.

A participant asked which program elements were the top priorities?

The Government answered that all program elements are the same priority and others could be added.

A participant asked if roadside enforcement included weight.

The Government answered that it did and added technology could possibly reduce the number of roadside stops for weight checks.

A participant wondered if the Government intended to use the results of the Federal Highway Administration (FHWA) study of the use of technology in freight manifests.

The Government stated in reply that they would use the results of the ongoing study.

One participant stated that inspections were in the SOW but not the legislation. He asked what was the relative priority of inspections, freight mobility and safety?

The Government answered that they had added safety inspections and that the priorities were equal.

One participant offered that truckers say parking is an issue. Technology could be used to make parking easier.

The Government said that was a good suggestion. There was a FMCSA-led study under way with a field test. It's called "Smart Park". The Government may consider integrating this effort with that study.

Question 2: Are the tasks durations and structure reasonable?

Question 3: Should the statements of work be restructured? If so, how?

This question generated a great amount of discussion. One common thread was ways to align the freight study and the FOTs. Some participants suggested that the freight study and FOTs be consolidated into one task. Others added that the study should be conducted first, a concept of operations developed and then the FOTs conducted. Further it was suggested that the freight study should be done first, decisions made on which FOTs to conduct, and then issue separate contracts for each FOT. It was generally agreed that this approach would be risky because it would be difficult to award separate contracts in time to obligate funding.

There was concern expressed that the need to obligate funds by September 30, 2006 may drive the bidders to risk proposing areas for FOTs before the inefficiency study was completed, resulting in widely divergent proposals which would be difficult for the government to evaluate.

Concern was expressed that aligning the evaluation team and the study team may result in a biased evaluation. Some agreed with this concern while others felt this potential problem could be overcome with close communication between the teams and strong management by the contractor.

While there was little consensus among participants on these matters the Government stated they would carefully consider all input. One participant suggested the Government and contractor form a joint steering committee to oversee the entire project. The Government stated it would consider that suggestion. The Government also stated it would consider the suggestion to require the contractor to submit a business plan detailing how they intend to manage the different project elements.

The following comments and questions represent the consideration of this question:

One participant suggested there seemed to be a disconnect between the Field Operational Test (FOT) and the freight study.

The Government answered that the draft SOWs were structured so that the FOT(s) would logically follow and test wireless solutions to problems identified in the freight inefficiency study.

A participant suggested the freight inefficiency study and the FOT should be one task. Another participant commented that the draft SOWs reflect this and continued by stating that awardees should do the study first then the FOT(s).

One participant suggested the Government should consider doing the study; decide which FOTs to conduct, and then do separate contracts for each FOT.

In answer to that suggestion another participant said we had to be practical and realize that to award the contracts and obligate the funds by September 30 vendors might take risks in proposing areas for FOTs before the study was completed. That could result in an "apples and oranges" situation.

In response to that comment the Government said they did not want vendors’ proposals to be drastically different. Bidders should have different approaches on the same topics.

A participant stated that the original structure still allowed the evaluation team and study team to collaborate, review and coordinate on the study and FOTs. This approach would need the evaluation teams' involvement early in the process.

In response to that comment, another participant stated that unless you're driving a predetermined solution, you should not allow the study team and the evaluation team to work together. He stated one example was a project that had the same supervisor and Contracting Officer but separate contracts. The resulting product failed.

A participant added that if the study and FOTs were joined, how could you not have a biased result?

The Government answered that there needed to be independence and careful consideration of how to avoid that issue. They would look to the contractor to manage that.

One participant suggested that the Government consider establishing a steering committee to guide all elements of the project. This committee could review progress, direct milestones and ensure coordination.

The Government stated they would examine that suggestion.

A participant suggested it would be difficult, if not impossible to cost out the FOTs without the results of the study. He was concerned about the Governments' ability to evaluate the proposals and award by September 30. Perhaps the basis for the evaluation should be the capability of the vendors to do the study and manage the FOTs.

The Government answered they would consider including assumptions and parameters in the solicitation to help bidders cost the FOTs.

A participant suggested the Government should allow industry to tell the Government the percentage of allocation each year that should be used for FOTs.

The Government answered they would consider that.

A participant asked if field tests had to be conducted for each element in the law.

The Government answered that the law required to conduct field tests "as appropriate" so the team needed to make reasonable calls on when to conduct FOTs and the size of FOTs. For example, since there was an active freight manifest study the Government may not want to do another one or at least delay the start of a related FOT until the current test is completed.

Question 4: Is the duration for the freight study appropriate?

The consensus among the participants was that the four months allowed to complete the study was too short. Nearly all participants felt that six to eight months would be needed.

Question 5: Should the field operational tests be one overall test with different parts covering the Program Elements, individual tests, or a hybrid?

The discussion on this question centered on two issues. First, the participants reached consensus and recommended that the Government should not break up the FOTs because if there are too many small ones, especially with separate contracts for each FOT, then it would not be economically feasible to bid on the individual contracts. Second, most participants felt that the FOTs should be structured around technologies that apply across most segments of the industry with the results of the FOTs addressing issues of interest to those segments.

The following comments and questions represent the discussion as this question was considered:

The Government started the conversation by stating that currently the study/FOTs and evaluations are separate. They asked if the SOWs should be restructured to reflect one SOW for the freight study and evaluation and the other SOWs for the conducting of the FOTs?

As noted above, one participant responded by stating he felt the group didn't want the Government to break up the FOTs because there would be too many small ones. If that happened and they did separate contract for each FOT there would not be enough dollars to encourage creative bidders. Also, the integration of the results of FOTs conducted under separate contracts would be difficult. He recommended that they conduct the study first and then request proposals for the FOTs, if that was possible.

That participant also asked, with general agreement from the other participants, if the study team, under the guidance of FMCSA be authorized to sub-contract FOTs?

The Government replied they would consider that suggestion.

One participant stated that since technology was highly variable across the trucking industry, the Government should consider testing applications that apply across the entire industry.

Another participant added, with uniform agreement by all present that for this effort to matter the results must be relevant to carriers' operations. He suggested the Government should be sure the language allowed flexibility in technical solutions.

Question 6: What is the best labor mix?

The participants did not identify all the specific skills needed to complete the study and FOTs, other than program management, wireless technology and trucking industry expertise.

The following comments and questions represent the discussion as this question was considered:

The Government asked if awardees could get letters of commitment from fleets and other team members. They wondered if developing long range relationships was feasible.

The participants agreed that this approach was possible.

A participant stated that getting letters of agreement with fleets was not as much an issue as with technology partners.

The Government suggested contractors could make a choice of partners and then add competencies as needed. Participants agreed that was possible.

One participant suggested that one could put together a core team and add expertise as needed based upon the results of the study.

Question 7: For the evaluators: what measures of effectiveness should be used to evaluate safety, productivity, and cost reduction improvements for the wireless technology solutions within the program elements?

Several potential measures of effectiveness were offered by the participants during a free-flowing discussion. The Government began the discussion by asking what measures of effectiveness were common to the industry. Participants suggested the following measures:

• Reduction in dead head miles (empty trailers).
• Out of route miles.
• Trucker's technical competence.
• Ways to avoid traffic.
• Loading and unloading times (reduction in wait times). The participant stated that the biggest problem is in receiving. Truckers represent extra storage space. One can wait in line at gates to docking facilities for hours, then more hours at the actual docks. Fatigued drivers do not log time waiting.
• Cost per mile. The total picture of cost is different for each supply chain. Ways technology can improve on-time delivery could be important. Equipment utilization (chassis moves per month). Missed delivery times; number of turns per day.
• Metrics by industry segments. We must be certain that improvements identified focus on "mom and pop" operations. Fuel costs, fuel productivity. Within the segments, there may be variations on commodities.
• Drivers' compensation could be a factor. Are they paid by the hour or by the mile?
• Compliance with regulations and the logging process should be addressed to show schedulers how many hours available does a specific driver have.
• Metrics could include driver productivity, how the vehicle is performing, which could include miles per gallon (mpg), and maintenance cost, a possible indicator of the driver's braking patterns, for example.
• Measurements will be different each of the commodities but we could look at things such as volume of trailer versus volume used, accident rates, on-time performance rates, missed shipments, and damage rates per shipment. We could also measure the sequence of loading and unloading, accuracy of deliveries, and downtime resulting from State operated regulatory compliance checks. We also need to look at the bottom line; what are the transportation costs.
• Other items measured could include administrative costs for verbal and data communications, and time at freight transfer points. Further we could measure stops per day, idling time, overtime, dispatcher-to-driver ratio, time-to-payment after proof of delivery, inventory reduction, amount of theft and pilfering, and route time reduction.
• We could measure downtime for vehicles, average time for inspections, asset management and ways to couple assets with available drivers for deliveries.
• Another area to measure could be quality of life for the drivers as measured by retention rates and time at home. We'd need to measure average age of drivers.

Question 8: Are there any propriety data issues regarding the data to be collected?

This question brought out some participants concerns about maintaining proprietary data as well as independent truckers' privacy. The Government stressed the data collected from the project would be used for research purposes. The following questions and comments represent the discussion surrounding this question:

The Government began the discussion by stating financial and operational data may be needed. Products would include the design of FOTs. Would operators and vendors be willing to disclose this data?

One participant suggested that as for the FOTs, they could disclose a high-level "wiring diagram" without specific design details. As for operational details, they could disclose how functional components interact, not such items as the internal codes and processing designs. However the team needed to be sensitive to participating companies operating data; for example, a specific company's trailer utilization ratio.

The Government stated that a possible answer to that concern may be that the Government would not receive information on company names.

The Government asked if industry inefficiency operational technology studies are proprietary.

Several participants answered yes. The results are not shared. They suggested that one inefficiency in the industry is the lack of flow of information between segments of the trucking industry, as well as the Government. For example, information on traffic congestion could be shared but normally was not.

One participant asked how the Government would protect the identity of carriers who participate in the FOTs.

The Government stated they would consider ways to do this.

One participant stated another concern was privacy. Privacy was very important to the small business, independent truckers. Most truckers were small businesses and did not want someone knowing where they were 24/7.

The Government responded that, for the project, the FMCSA was not interested in collecting proprietary business operations data specific to a trucking company.

Another participant added that data could also be used in liability issues involving accidents and other things against truckers.

Question 9: Are there cost-sharing opportunities?

The Government initiated the conversation by stating that cost sharing was not a requirement for this project. However, the Government encouraged voluntary cost sharing initiatives. The only response from several of the participants was that the Government should not put too much weight on cost-sharing, especially in the evaluation segment.

Conclusion

The Government concluded the session by thanking all of the participants for their considerable and valuable input. They stated they would seriously consider the input from the session and that they would make the session's report available to the public. They stated that they may conduct a pre-proposal conference. They further stated that any future questions and comments should be submitted to the Contracting Officer, Ruby Mixon.

Attachments

February 3, 2006

Mr. Jeff Loftus
Federal Motor Carrier Safety Administration

Subject: SEC. 5503. MOTOR CARRIER EFFICIENCY STUDY.

Dear Mr. Loftus,

We have received the Draft Statement of Work for the "Motor Carrier Efficiency Study" and wish to offer a few comments and suggestions for structuring this work.

By way of background, it appears that FMCSA is considering two separate procurements related to this work:

1. Selection of a "conductor team", or a team of industry stakeholders who would perform a study to identify freight inefficiencies, propose wireless based solutions, and then conduct a field test or demonstration to document the costs and benefits of the proposed solutions, and:
2. an Independent Evaluator who would work with FMCSA to ensure the field tests and demonstrations produce useful data, lead collection and analysis of the data, and summarize costs and benefits of proposed wireless solutions.

Our concern is with the scope of work for the first or "primary" contract with a conductor team. Specifically, the draft SOW calls for conducting a study of inefficiencies (or opportunities) in the commercial motor carrier industry where wireless technology could be of benefit, with candidate study areas being:

• Fuel monitoring and management systems
• Radio frequency identification technology
• Electronic manifest systems
• Cargo theft prevention
• Roadside safety inspection systems

The SOW allows for the possibility that other areas of inefficiency might also be identified during this first study task. The conductor team would then further identify wireless based technologies and commercial solutions that address the inefficiencies and opportunities. After completing this first phase of work, the contractor would then work with FMCSA to plan and execute a field operational test or demonstration in order to showcase the proposed solutions and to quantify costs and benefits.

The challenge in responding to this scope of work is that the resources, level of effort and indeed optimal teaming partners to complete the second phase of the work (executing an field test) are very much dependant on the outcome of the first phase of work (the inefficiencies study). This makes it very difficult to assemble an appropriate team to respond to the entire scope of work, and to develop a cost proposal for completing the work. For example, the fleet partners, equipment suppliers and industry experts that could be assembled to address inefficiencies or opportunities related to fuel monitoring and management systems are arguably quite different than those focused on say roadside inspection systems. While a collection of team partners could be assembled that would be capable of executing a field operational test in any or all of the candidate areas of study, information needed to complete a cost proposal would still be lacking since the results of the "inefficiency study" would not have been completed, (also the Team would be unnecessarily large and complex).

Alternative Study Structure: To address this issue, we offer the following alternatives:

1. The Industry Inefficiency Study (task 1) could be a separately priced task within the scope of work for the Conductor Team. The contract for the Conductor Team would be awarded based on the bidder's qualifications and approach for conducting the study, as well as on their overall resources, breadth and depth of capabilities for conducting technology demonstrations in general, and on pricing for just the first phase of work (the inefficiency study).
2. The Industry Inefficiency Study could be an entirely separate procurement. A solicitation for the Conductor Team would not be issued until the results from the inefficiency study were completed. The solicitation for the Conductor Team could then be properly scoped so as to more clearly define the specific technologies to be tested and demonstrated, the objectives and goals for the FOT, data to be gathered and other specifics that would be identified during the execution of the Inefficiency Study. This way, prospective bidders for the Conductor Team would have a more solid understanding of FMCSA's needs, and would be able to assemble an efficient team of industry stakeholders in order to execute the work.
3. The Industry Inefficiency Study could become part of the scope of work for the Independent Evaluator, rather for the Conductor Team. Again, the issuance of a solicitation for the Conductor Team would be delayed until the results of the Industry Inefficiency Study were essentially complete.

We submit these suggestions recognizing that we may not have a full understanding of FMCSA's goals, constraints, resources and/or other related programs that may impact the manner or context in which this Study is to be conducted. We offer these ideas in the spirit of promoting an efficient, fair, and competitive procurement process.

Potential Measures of Effectiveness for Evaluation of The Motor Carrier Efficiency Study (MCES)

The following are suggestions, not an exhaustive list. The Measures of Effectiveness (MOE's) follow the Program Elements in Paragraph 2.4 of the MCES Draft SOW:

Program Element #1 – Fuel monitoring and management systems

• Fuel savings per trip
• Emissions reduction per trip
• Potential reductions in fuel-related accidents/incidents
• Reduction in lost time from unintended fuel outages

Program Element #2 – Radio frequency identification technology

• Reductions in total process time through error reduction, reducing time spent matching cargo, trailers, and tractors
• Reductions in truck queue times at ports, intermodal facilities and terminals
• Reductions in errors in matching equipment and cargo
• Reductions in human resources (phone calls, paperwork, etc.) due to automated information provided by RFID
• Improved utilization of equipment as a result of more accurate and timely truck and shipment identification
• Increased customer confidence in integrity of shipping process

Program Element #3 – Electronic manifest systems

• The table below presents an overview of the current draft evaluation approach that was designed for the independent evaluation of FHWA's Electronic Freight Management (EFM) program, formerly known as the Electronic Freight Manifest program. To the degree this approach can be applied to FMCSA's testing of electronic freight manifests, it could facilitate an "apples to apples" comparison of potentially similar technology components being tested by the two USDOT agencies.

Table 1. CEFM Evaluation Objectives and Learning Outcomes

Specific measures of effectiveness within these overall objectives should include:

• Reduction in erroneous moves and lost cargo via more accurate information
• Reduced traffic congestion through reductions in erroneous moves and reductions in truck dwell times at nodes.
• Reduced overall administrative costs from automated information.
• Reduced costs associated with erroneous billings.
• Increased schedule adherence and on time performance.

Program Element #4 – Cargo theft prevention

• Reductions in cargo pilferage or theft
• Enhanced driver security
• Increased security against terrorism through reduced risk of tampering along the supply chain
• Reductions in misuse of equipment by customers and supply chain partners
• Increased customer satisfaction

Program Element #5 –Roadside safety inspection systems

• Further reduction in stops for firms with high compliance records
• Higher productivity for safety inspectors
• Measurable reduction in equipment-based incidents/accidents
• Improvement in compliance rates by firms with less-than-the-best compliance records

Industry Capabilities Meeting
Participant List
2/9/2006