Detailed Information on the
Bureau of Transportation Statistics Assessment

Developing a new, more systematic performance measure that identifies the program's primary users/customers and solicits feedback from those users.

Addressing identified management deficiencies, especially the need to fill formal leadership positions.

Bridging the program's connection to Departmental goals while maintaining an adherence to federal statistical performance dimensions

Measure: Number of Congressional and governmental contacts for BTS information

Explanation:BTS is a major resource for transportation information for Congress and other government agencies. The number of government and Congressional contacts measures BTS' strategic goal of relevance which is the degree to which products and services are useful and responsive to users' needs. BTS' long term goal to increase awareness of its data and products as well as usefulness of its products is evaluated by this measure. Since BTS is a process oriented agency an output measure has been developed as a proxy outcome measure.

Measure: Average daily number of unique visitors to TranStats data warehouse

Explanation:The TranStats data warehouse provides access to information on all modes of transportation. The average number of visitors to TranStats measures BTS' strategic goal of relevance which is the degree to which products and services are useful and responsive to users' needs. BTS' long term goal to increase the use of its data and products is evaluated by this measure. Since BTS is a process oriented agency, an output measure has been developed as a proxy outcome measure.

Measure: Commodity Flow Survey response rate

Explanation:The survey response rate measures BTS' strategic goal of accuracy which is the correctness, validity, and reliability of data and information products. The survey response rate measures BTS' long term goal to collect and provide accurate, valid, and reliable data. Since BTS is a process oriented agency an output measure has been developed as a proxy outcome measure. The program has worked aggressively to maintain a response rate in the low 80% range for the CFS by making additional investments in the data collection process including: conducting precanvass operations in advance of the survey to obtain better contact information, verify eligibility, and removal of duplicates; redesigning the survey form, instructions in response to cognitive interviewing results; and offering an electronic reporting option.

Measure: Omnibus Household Survey response rate

Explanation:The survey response rate measures BTS' strategic goal of accuracy which is the correctness, validity, and reliability of data and information products. A high survey response rate reduces the potential for non-bias response or inaccurate data. BTS' long term goal is to increase the survey response rate thereby improving BTS' ability to collect and provide accurate, vaild, and reliable data. Since BTS is a process oriented agency an output measure has been developed as a proxy outcome measure. The Omnibus Survey is unique in that the survey is new in every iteration. Because this instrument is designed to be a quick response tool on current issues of importance to the Department, the questions are revised with each issuance. The Omnibus Survey should be thought of as 12 separate surveys rather than a monthly survey which instead indicates a consistent, ongoing data collection effort. And because the topics and questions fluctuate, the target audience actually changes with each issuance. New survey instruments with a changing pool of target responders means significantly less precanvassing can be done to reduce incorrect contact information, less ability to test question presentation and response formats to improve usability by the respondents, and generally less a advanced instrument. The trade-off is by design - the Omnibus is intended to generate quick-responses on issues of immediate importance and must permit lower response rates in order to meet is intended purpose. Therefore, a target response rate around 50% is ambitious given the nature of the survey design.

Measure: Percent of airline traffic data released on time

Explanation:Monthly BTS publishes data on the airline travel and traffic levels. The percent of airline traffic data released on time measures BTS' strategic goal of timely information releases. BTS' long term goal is to reduce the lag time in data reporting so decision makers have a "real time" view of the system and factors affecting it. This performance measure evaluates BTS strategic and long term goals of improved timeliness. Since BTS is a process oriented agency an output measure has been developed.

Measure: Percent of airline financial data released on time

Explanation:Quarterly BTS publishes data on the airline industry financial profits and losses. The percent of airline financial data released on time measures BTS' strategic goal of timely information releases. BTS' long term goal is to reduce the lag time in data reporting so decision makers have a "real time" view of the system and factors affecting it. This performance measure evaluates BTS strategic and long term goals of improved timeliness. Since BTS is a process oriented agency an output measure has been developed.

Measure: Annual production costs for major reference products

Explanation:This production costs for major reference products measures BTS' strategic goal of cost which is the dollar amount used to produce data products and services. The annual performance measure evaluates BTS goal of reducing the cost of recurring programs: Transportation Statistics Annual Report, National Transportation Statistics, and the Pocket Guide to Transportation Statistics.

Measure: Annual cost of collecting international freight data

Explanation:The annual costs for collecting international freight data measures BTS' strategic goal of cost which is the dollar amount used to produce data products and services. The annual performance measure evaluates BTS goal of reducing the cost of recurring programs. BTS continues to seek ways to reduce the cost of collecting the international freight data.

Measure: Number of National Transportation Atlas Databases distributed

Explanation:BTS distributes its National Atlas Transportation database to its user base via CD-ROM. The number of databases distributed measures BTS' strategic goal of dissemination which is the availability, accessibility, and distribution of information products and services. BTS' annual goal of increasing the user base of the Atlas database as well as making products easily accessible is evaluated by this measure. BTS using an output measure since it is a process oriented agency.

Measure: Number of datasets downloaded from TranStats

Explanation:TranStats is the only transportation statistics portal in the United States. User downloads from the TranStats database are a major indication of utility in the transportation community. This measures BTS' strategic goal of dissemination which is the availability, accessibility, and distribution of information products and services.

Is the program purpose clear?

Explanation: The Bureau of Transportation Statistics (BTS) is a Federal statistical agency within the Research and Innovative Technology Administration (RITA) of the Department of Transportation (DOT). The purpose of the program, as stated in its strategic plan, is to serve as the principal transportation statistics agency providing accurate, relevant, and timely transportation data that will guide actions and policies to improve the transportation system in the United States. As authorized by law, BTS is charged with conducting and supporting statistical activities to improve the effectiveness, efficiency and quality of transportation information in the United States. BTS compiles a variety of transportation statistics, produces the Transportation Statistics Annual Report, issues statistical guidelines, maintains a transportation database warehouse (TranStats), maintains the National Transportation Library, produces the Transportation Atlas Database, protects confidential information, collects airline information, and makes the transportation information that the agency publishes readily accessible.

Evidence: 1. The Bureau of Transportation Statistics (BTS) Strategic Plan: http://www.bts.gov/publications/strategic_plan/2003_2008/pdf/entire.pdf 2. 49 U.S.C. 111

Does the program address a specific and existing problem, interest, or need?

Explanation: BTS's mission is to lead in developing transportation data and information of high quality, and to advance their effective use in both public and private transportation decision making. BTS was authorized in 1992 as the only Federal statisticial agency to provides a variety of transportation statistics to guide decisions and policies for improving the transportation system in the United States. BTS programs operate to meet the needs identified in the agency's authorizing legislation: BTS ensures that these needs are met through transportation statistical compilation reference documents, data collection, confidentiality, data quality, airline information, transportation databases, national transportation library, geospatial data, and information product dissemination.

Evidence: 1. BTS Strategic Plan http://www.bts.gov/publications/strategic_plan/2003_2008/pdf/entire.pdf 2. 49 U.S.C. 111

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: BTS, as a national program, is designed to reduce duplication at the Federal, State, and local levels. The BTS program focuses on intermodal and multi-modal transportation which means that BTS is providing analysis of transportation data that bridges geographic and modal lines. No other Federal, State, or local agencies are conducting similar analysis. In addition to its unique mission, BTS works with the DOT, the Office of Management and Budget (OMB), the Transportation Research Board, and the American Association of State Highway and Transportation Officials to issue Federal Register notices that ensure its programs are efficient, address unique topics and populations, and are not redundant with other government and private data collection efforts. OMB has approved BTS' data collection programs and affirmed that the agency's programs are not duplicative.

Evidence: 1. 49 USC 111 2. Federal Register notices for BTS information collection initiatives. National Household Travel Survey (64 FR 56241 and 64 FR 27852); Commodity Flow Survey (66 FR 46599); Customer Satisfaction Surveys Program (70 FR 41814); 3. OMB approved data collections: http://www.whitehouse.gov/omb/library/OMBINV.STATE.DOT.html#DOT [prefix numbers 2138 and 2139]

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: BTS' program design is free of major flaws. BTS programs have been reviewed and evaluated independently by the National Academy of Science's Transportation Research Board (TRB). Based on TRB's recommendations, BTS consults and gets advice from its Advisory Committee on Transportation Statistics (ACTS) for program effectiveness. Proposals for new information products are reviewed for effectiveness and quality of design through BTS' Product Scoping Paper review process. This ensures that major flaws are reduced before the information product work begins. Although the BTS program areas, strategic plan, and budgeting process are aligned, staff are organized into functional areas. Thus, most program areas involve staff from multiple functional areas. BTS is currently reorganizing to address this issue. There is no strong evidence that another program design would be more effective or efficient.

Evidence: 1. National Research Council Transportation Review Board Evaluations: Bureau of Transportation Statistics: Priorities for the Future(http://ntl.bts.gov/lib/6000/6100/6124/priorities.pdf), Measuring Personal Travel and Goods Movements: A Review of the Bureau of Transportation Statistics' Surveys (http://darwin.nap.edu/html/SR277/SR277.pdf), A Concept for a National Freight Data Program, (http://fermat.nap.edu/html/SR276/SR276.pdf). 2. BTS Product Scoping Paper review process

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Output from BTS programs is targeted to meet the requirement of law and the intended beneficiaries in the transportation community. To ensure that BTS' limited resources are being put to the best possible use, the agency engages in the ongoing solicitation and/or analysis of stakeholder feedback, obtained through forums including conferences, listening sessions, and special reports of the Transportation Research Board and the National Research Council. Extensive outreach to BTS stakeholders and customers over the past five years has garnered feedback that has been directly put to use in BTS' organizational design, helping to transform it from an internal process-driven statistical agency to one that is customer focused. Program prioritization has led to the discontinuation of less critical programs, in favor of shifting more resources to the priority areas of freight and airline data. BTS product design itself has changed, with a marked shift over the past two years away from printed reports toward more interactive, accessible, and timely statistical product releases on the BTS website.

Evidence: 1. Measuring Personal Travel and Goods Movements: A Review of the Bureau of Transportation Statistics' Surveys (http://darwin.nap.edu/html/SR277/SR277.pdf ) 2. Geospatial Information Infrastructure for Transportation Organizations: Toward a Foundation for Improved Decision Making (http://trb.org/publications/conf/CP31spatialinfo.pdf ) 3. A Concept for a National Freight Data Program(http://fermat.nap.edu/html/SR276/SR276.pdf) 4. Commodity Flow Survey conference, July 2005 (http://www.trb.org/publications/circulars/ec088.pdf) 5. Personal travel data users meetings: Data for Understanding our Nation's Travel: National Household travel Survey Conference, November 2004 http://trb.org/Conferences/NHTS/Program.pdf

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: BTS has long-term performance measures that focus on outcomes and reflect the common performance measures of statistical agencies. The Federal statistical agencies represented on the Interagency Council on Statistical Policy (ICSP) have collaborated on developing an initial set of common performance standards to maintain the quality of their data or information products as well as setting high performance standards. The agencies have agreed that there are two general areas of focus that are key to measuring and monitoring statistical programs. The first area of focus is Product Quality, encompassing the traditional dimensions of relevance, accuracy, and timeliness. The second area of focus is Program Performance, encompassing the dimensions of cost, dissemination, and mission achievement. The BTS Strategic Goals are aligned with the recommendations of the Interagency Council on Statistical Policy and supported by seven measures of long-term performance that are consistent with the recommendations of the Interagency Council on Statistical Policy. The long term measures focus on the strategic goals of a) relevance (the degree to which products and services are useful and responsive to the needs of customers and stakeholders), b) accuracy (the correctness, validity, and reliability of data and information products), c) timeliness (the timing of information releases), d) cost (the dollar amount used to produce data products and services), e) dissemination (the availability, accessibility, and distribution of products and services, and f) mission achievement (the effectiveness of BTS information programs in satisfying customer and stakeholders). Since BTS is a process oriented agency it has used some output measures as a proxy for outcome measures.

Evidence: 1. Analytical Perspectives, Budget of the United States Government, FY 2006, Section 4, Strengthening Federal Statistics 2. The BTS Strategic Plan (http://www.bts.gov/publications/strategic_plan/2003_2008/pdf/entire.pdf) presents the strategic goals and the measures used to support the strategic goals. 3. The BTS Performance Report (http://www.bts.gov/publications/performance_report/) contains the measures, targets, timeframes, and actual performance data for each of the strategic goals.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Each of the BTS long-term quantitative performance measures has ambitious targets and timeframes and baseline data. For example, for Congressional and governmental contacts, the target for 2008 requires that BTS more than quadruple the number to 350 in 2008 and aim to increase that number to 400 by 2010. For daily visitors to the TranStats data warehouse, the goal requires that BTS increase the number of unique daily visitors from a baseline level of 565 in 2003 to 950 in 2008 and reach 1000 in 2010. There are two long-term measures of Accuracy both require that BTS exceed a standard of statistical survey quality.

Evidence: 1. The BTS Performance Report (http://www.bts.gov/publications/performance_report/) lists the measures, targets, and timeframes, for each of the BTS strategic goals.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The Federal statistical agencies represented on the Interagency Council on Statistical Policy (ICSP) have collaborated on developing an initial set of common performance standards to maintain the quality of their data or information products as well as setting high performance standards. The agencies have agreed that there are two general areas of focus that are key to measuring and monitoring statistical programs. The first area of focus is Product Quality, encompassing the traditional dimensions of relevance, accuracy, and timeliness. The second area of focus is Program Performance, encompassing the dimensions of cost, dissemination, and mission achievement. The BTS Strategic Goals are aligned with the recommendations of the Interagency Council on Statistical Policy and supported by five measures of annual performance. Consistent with the recommendations of the Interagency Council on Statistical Policy, the annual measures focus on the strategic goals of a) relevance (the degree to which products and services are useful and responsive to the needs of customers and stakeholders), b) accuracy (the correctness, validity, and reliability of data and information products), c) timeliness (the timing of information releases), d) cost (the dollar amount used to produce data products and services) e) dissemination (the availability, accessibility, and distribution of products and services, and f) mission achievement (the effectiveness of BTS information programs in satisfying customer and stakeholders). Since BTS is a process oriented agency it has used some output measures as a proxy for outcome measures.

Evidence: 1. The BTS Strategic Plan (http://www.bts.gov/publications/strategic_plan/2003_2008/pdf/entire.pdf) presents the strategic goals and the measures used to support the strategic goals. 2. The BTS Performance Report (http://www.bts.gov/publications/performance_report/) contains the measures, targets, timeframes, and actual performance data for each of the strategic goals.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Each of the BTS annual performance measures has a baseline accompanied by ambitious targets that are achievable. For example, by finding production efficiencies, the targets for the cost of producing major published reference products require that the cost be reduced annually from a baseline of $802,500 in 2001 to a cost of $475,000 in 2008. For the number of National Transportation Atlas Database CD-ROMs distributed annually, the targets for this measure require a broadening of the customer base for this product incrementally from a baseline of 997 in 2004 to 3,500 in 2008.

Evidence: 1. The BTS Performance Report (http://www.bts.gov/publications/performance_report/) lists the measures, targets, and timeframes, for each of the BTS strategic goals.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: BTS' partners include other agencies within DOT, other Federal agencies such as the Census Bureau and the Transportation Security Administration, the Transportation Research Board, and contractors. For example, BTS regularly consults with DOT's Office of the Secretary and the airline partners on the Aviation Data Modernization Rulemaking. This interaction and feedback ensures a continuous commitment to BTS's airline statistics program goals and promotes the BTS goals of relevance, accuracy, and efficiency. The Census Bureau is BTS's cost-sharing agency partner in International Freight data and the Commodity Flow Survey. Through memoranda of agreement, the Census Bureau is committed to meeting performance delivery dates as well as providing the specified content of the survey. BTS ensures commitment to its work by requiring its contractors to conduct quality assurance and commit to the long term goals of the various surveys.

Evidence: (1) 2005 Omnibus Statement of Work; (2) Memoranda of Agreement with Federal Railroad Administration and Transportation Security Administration; (3) Airline data reporting requirements are specified in the Uniform System of Accounts and Reports for Large Certificated Air Carriers (14 CFR 241, available at http://www.bts.gov/laws_and_regulations/docs/Part241cy2000.pdf); (4) The BTS schedule of upcoming data releases is available at http://www.bts.gov/upcoming_data_releases/

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: BTS has had, and continues to have, high quality, independent evaluations of its effectiveness and relevance. An assessment of comprehensive scope was conducted by the National Research Council to examine BTS as a whole, focusing especially on quality, relevance, and staffing skills. Since then, external independent evaluations have focused on major components of BTS programs. DOT's Office of the Inspector General (OIG) has over the years conducted audits of BTS airline data collections. These audits are sufficiently rigorous and comprehensive to provide information on the effectiveness and relevance of the entire airline data program problems, interests, and needs. A team of independent experts from the Transportation Research Board and the Committee on National Statistics have jointly reviewed BTS's survey programs. Their report gives detailed recommendations on ways to improve three of BTS's survey programs; its travel survey, Omnibus survey, and freight survey. BTS continues to undergo regular review and evaluation. In FY 2006, the Government Accountability Office is conducting an audit to examine its data quality and the usefulness of its products as part of a review of RITA.

Evidence: 1. The Bureau of Transportation Statistics: Priorities for the Future, National Research Council, 1997 (http://www.bts.gov/publications/bts_priorities_for_the_future/pdf/entire.pdf) 2. Office of the Inspector-General audits of Air Carrier Flight Delays and Cancellations CR-2000-112 (http://www.oig.dot.gov/StreamFile?file=/data/pdfdocs/cr2000112.pdf) 3. Measuring Personal Travel and Goods Movement, Transportation Research Board/Committee on National Statistics (http://trb.org/publications/sr/sr277.pdf) 4. Notification of GAO audit, "Review of Department of Transportation's Research and Innovative Technology Administration," Engagement code 542079 (November 16, 2005)

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The BTS annual budget submissions are not explicity tied to the program or agency's strategic plan or performance goals. The FY 2007 BTS budget describes the 2007 budget request for BTS and ties to DOT's Strategic Outcomes, but does not make explicit the links between DOT Performance Goals or the BTS strategic goals and performance measures.

Evidence: 1. BTS FY 2007 Budget Estimates in Exhibit IV-1 "Budget Request by Strategic Objective and Performance Goal."

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Following up on the recommendations of the National Research Council report, the agency took steps to increase its statistical and transportation specialist staff levels and to emphasize data quality. BTS also took a number of steps in response to the 2003 TRB/Committee on National Statistics report such as improving the content of its commodity flow survey. BTS updated its Strategic Plan in 2006 to incorporate guidance from the Interagency Council on Statistical Policy (ICSP). The most significant change was made by adding the ICSP directive for cost into the agency's strategic focus. The cost dimension indicates that "quantitative measure of the dollar amount used to produce data products and services should be applied within statistical programs.

Evidence: 1. The Bureau of Transportation Statistics: Priorities for the Future, National Research Council, 1997; 2. Measuring Personal Travel and Goods Movement, Transportation Research Board/Committee on National Statistics, 2003 (http://trb.org/publications/sr/sr277.pdf); 3. Strengthening Federal Statistics, Crosscutting Programs, Analytical Perspectives (Section 12), Budget of the United States Government, Fiscal Year 2007, pp. 35-42. (http://a257.g.akamaitech.net/7/257/2422/06feb20061000/www.gpoaccess.gov/usbudget/fy07/pdf/spec.pdf); 4. The BTS Performance Report (http://www.bts.gov/publications/performance_report/) lists the measures, targets, and timeframes, for each of the BTS strategic goals; 5. The BTS Strategic Plan (http://www.bts.gov/publications/strategic_plan/2003_2008/pdf/entire.pdf) presents the strategic goals and the measures used to support the strategic goals.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: BTS collects timely and credible performance information and uses that information to improve the management of its programs. For example, the BTS Director determined that publishing airline data on time for users was a critical performance element for the agency. Using timely performance data, BTS has been able to significantly improve the performance of this particular program and meet its annual performance goals. Beginning in 2006, BTS began publishing an annual performance report so that all managers have access to the agency's performance data and can improve the management of BTS programs. In addition, BTS uses feedback from its key partners to adjust program priorities, allocate resources, take appropriate management action, and assess progress on key program activities. BTS has relied heavily on the advice of its stakeholders in the design and management of its survey programs.

Evidence: 1. BTS Performance report (http://www.bts.gov/publications/performance_report/). 2. RITA-BTS AVIATION DATA STAKEHOLDERS AND USES OF THE DATA, internal document, March 13, 2006. 3. Measuring Personal Travel and Goods Movement, Transportation Research Board/Committee on National Statistics, [http://trb.org/publications/sr/sr277.pdf]

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and employees are held accountable for cost, schedule, and performance results through project plans which delineate the deadlines and deliverables required. Projects are reviewed and approved for purpose, mandated need, schedule, task specifications and assessed for funds through BTS project plan review process. The project plans are then tied to the performance reviews of all employees, including management. Key project deadlines appear in the business results section of employee performance reports. Contractors are also held accountable through deliverable requirements in task orders and, in the case of government partners, memoranda of understanding. All the requirements about the contents, quality, timely delivery and other conditions are well documented in the contracts and are monitored on an on-going basis. Contractors are also mandated to work with BTS to show continuous performance improvement. For example, for the Omnibus Survey, the first contract required a response rate of at least a 30% in the first month of the first year of data collection. The contractor was expected to show improvement in the response rate over the course of that first year. Afterwards, the contractor is required to sustain an average response rate over a year's period which is no lower than the "optimum" response rate.

Evidence: 1. BTS Project Plan review process. See "Project Plan" template; 2. BTS' Product Scoping Paper review process. See "Product Scoping Paper"; See the "Information Product Review and Approval" form; 3. Performance Management Program, Bureau of Transportation Statistics, internal document. 4. 2005 Omnibus Statement of Work, Section H

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: BTS currently has processes in place to ensure that funds are administered efficiently, but erroneous payments were made in fiscal year 2004. To ensure that funds are obligated in a timely manner and spent for intended purpose, BTS program managers develop an annual operating plan and the budget execution plans for the agency once BTS receives its annual allocation from the Highway Trust Fund. Once these plans are approved by the Administrator of RITA, program staff may begin executing their program funds. For contracting and reimbursable transactions, a procurement request is submitted to the relevant approving officials within RITA with final approval by the Finance Office to assure that the purchase is within budget and pertains to the agency's mission and the Department's goals. Once the procurement request is awarded, the project managers continue to monitor the contract. As part of the President's Management Agenda, in FY 2004 the Department conducted an audit to identify erroneous and duplicative payments made to DOT vendors. The final audit revealed that in the BTS account, two erroneous payments totaling $91,000 had occurred. One vendor had received a duplicative payment and one vendor had received an overpayment. These funds were recovered and the Department accounting system executed a new module to prevent repeat payments from occurring. As a result, BTS is less likely to incur erroneous payments in the future.

Evidence: 1. Copy of SF133 from EOY 2005 2. RITA Action Plan to Resolve FY 05 Audit Material Weaknesses and Findings. Monthly.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: BTS has procedures to ensure efficiency and cost effectiveness. BTS uses competitive bidding to secure contractors for the data collection and other related statistical work. Contractors are required to compete based on both the quality of their proposal and cost. This provides the best value to the government. BTS uses two efficiency measures to manage the cost efficiency for their major reference products and freight data. In addition, BTS has promoted efficiencies using information technology. For example, BTS encourages the use of e-filing for air carrier data reporting in lieu of hardcopy reports. E-filing has improved the overall efficiency of the BTS aviation program by reducing the need for data entry to input airline data. The e-filing is expected to result in improved accuracy, timeliness, and cost savings. BTS is developing a new aviation data processing system that is projected to provide an annual cost savings of $250,000 and a savings of $1,500,000 over the six year life cycle. There are currently no A-76 competitions planned for BTS.

Evidence: 1. DOT Business Opportunities and Vendor Information, http://www.dot.gov/ost/m60/busopven.htm; RFP for Omnibus Household Survey, http://www.fbodaily.com/cbd/archive/2001/02(February)/28-Feb-2001/rsol006.0htm; 2. "Life Cycle Cost Analysis" in Exhibit 300, "BTS Mid Tier Server/Airline Report Data Information System".

Does the program collaborate and coordinate effectively with related programs?

Explanation: BTS has built effective and productive working relationships with related organizations within and outside of DOT. Within DOT, BTS collaborates and coordinates with the Office of the Secretary of Transportation on aviation policy issues such as the Aviation Data Modernization initiative and the Air Transportation Safety and System Stabilization Act. BTS is cooperating with the Federal Railroad Administration and the rail industry in the development and testing of a rail close calls reporting system to improve transportation safety. BTS is also a member of the Federal Interagency Council on Statistical Policy, the Federal Geographic Data Committee, and spearheaded the development of data exchange standards for transportation within the National Spatial Data Infrastructure. As a member of these groups, BTS ensures that its statistical programs are consistent with other Federal statistical and data programs. Finally, BTS collaborates with non-federal partners such as the American Association of State Highway and Transportation Officials and the American Public Transportation Association which are partners in the Survey of State Funding for Public Transportation.

Evidence: 1. Aviation Data Modernization (Docket OST-98-4043 at http://dms.dot.gov), http://dmses.dot.gov/docimages/p73/200517.pdf, Intra-Alaska Bush Mail Rate Making (Docket OST-2003-14694), http://dmses.dot.gov/docimages/pdf92/335008_web.pdf, Air Transportation Safety and System Stabilization Act compensation payments, http://caselaw.lp.findlaw.com/data2/circs/dc/041436a.pdf; 2. Federal Railroad Administration MOU August 2005; 3. Something from Interagency Statistical Policy group- maybe membership? 4. 2003 AASHTO Survey, http://downloads.transportation.org/scopt-funding_survey.pdf .

Does the program use strong financial management practices?

Explanation: BTS uses effective financial management practices in administering program funds. For example, each program uses annual operating plans and budget execution plans to manage its programs. Based on A-123 guidance, RITA has initiated an internal control assessment and BTS is currently rated "effective" in five areas: Control Environment, Risk Assessment, Control Activities, Information & Communications, and Monitoring. In addition, each month RITA reports to DOT on intragovernmental transactions and non-compliance for reconciling reimbursable agreements within and outside DOT and failure to use correct trading partner codes. The DOT IG audits BTS annually as part of a larger audit and has not found any material weaknesses for BTS. BTS did have erroneous payments in fiscal year 2004 totaling $91,000, but corrective actions have been taken to avoid future erroneous payments.

Evidence: 1. The monthly OMB metrics on the DOT CFO webpage: http://communities.dot.gov/uploadedFiles/0518HHC87447.xls. 2. Zip file containing RITA's documentation of major financial and other processes. 3. RITA Internal Control Self Assessment.xls.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: BTS has not addressed a major management deficiency in recent years. BTS has been without a permanent Director since October 2002. It is unclear when this position will be filled with a permanent leader. A lack of formal leadership makes it more difficult for BTS to address management deficiencies or strategic planning. BTS has still been able to address some issues in the last few years. To improve the management of resources, DOT required BTS to shift its organization and budget structure from a process-based focus to a program-based focus. BTS undertook program planning to design the purpose and scope of each program, identify primary customers and key projects within each one, and define the direction of each program. Since the creation of the BTS program structure, managers now have the ability to track resources to programs in addition to tracking resources to the office structure. To better integrate performance data into BTS planning and management, BTS revised its strategic plan and developed annual performance plans. Finally, RITA has developed to new protocols for requesting and approving all obligations thereby ensuring better financial management.

Evidence: 1. Program planning templates for Phase I and Phase II of planning. Participant lists and leads for each Program Planning team. Multiple drafts, team meeting minutes, and final versions of Phase I planning that incorporated the feedback of the Executive Management team. Program summaries distributed and used as primary agenda items for the Advisory Committee Meeting in 2003. 2. 2004-2007 Departmental, OMB and Congressional budget submissions. Internal spreadsheets used to track program resources. BTS Performance Plan.Project planning template. A collection of project plans developed by project leads in 2004. 3. Transcipts and notes from All-Hands meetings. Minutes and memos from the Employee Quality Council.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: BTS has made progress in achieving its long-term performance goals. BTS exceeded the FY 2005 target for five out of the eight long term goals. While, BTS did not meet the established targets for the Omnibus Survey response rate, the Airline Financial Data releases, and number of datasets downloaded from TranStats, BTS made notable achievement and has taken action to improve future outcomes. For the Omnibus Household Survey response rate survey interviewer training has placed greater emphasis on the ability of the interviewer to clearly understand and convey the purpose of the survey in a conversational manner. To address the airline financial data releases, BTS has introduced a data processing and dissemination monitoring and reminder system. The TranStats program has been refocused and the decrease of the number of datasets downloaded in FY 2005 reflected this strategic re-positioning. BTS will consider adjusting future year targets to reflect this reprioritization.

Evidence: 1, The BTS Strategic Plan presents the strategic goals and the measures used to support the strategic goals. The BTS Strategic Plan, 2000-2005 is posted on the BTS website at http://www.bts.gov/publications/strategic_plan/entire.pdf. The BTS Strategic Plan, 2003-2008 is posted on the BTS website at http://www.bts.gov/publications/strategic_plan/2003_2008/pdf/entire.pdf. 2. The BTS Performance Report contains the measures, targets, timeframes, and actual performance data for each of the strategic goals. The BTS Performance Report, 2005 is posted on the BTS website at http://www.bts.gov/publications/performance_report/.

Does the program (including program partners) achieve its annual performance goals?

Explanation: BTS has made progress in achieving its annual performance goals. BTS exceeded the targets for five of its six annual measures. BTS holds contractors and partners accountable for performance deliverables in their task orders; therefore partners are critical to the ability of BTS to meets its performance goals.

Evidence: 1, The BTS Strategic Plan presents the strategic goals and the measures used to support the strategic goals. The BTS Strategic Plan, 2000-2005 is posted on the BTS website at http://www.bts.gov/publications/strategic_plan/entire.pdf. The BTS Strategic Plan, 2003-2008 is posted on the BTS website at http://www.bts.gov/publications/strategic_plan/2003_2008/pdf/entire.pdf. 2. The BTS Performance Report contains the measures, targets, timeframes, and actual performance data for each of the strategic goals. The BTS Performance Report, 2005 is posted on the BTS website at http://www.bts.gov/publications/performance_report/.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Since BTS funding levels have decreased slightly in recent years, BTS cost efficiency is a continuous goal. In 2005, BTS exceeded the 2005 target for all of its efficiency goals: Production Cost Efficiency for Major Reference Products and Cost Efficiency for Acquiring International Freight Data. For the Transportation Statistics Annual Report, between FY 2004 and FY 2005, total costs were reduced 2.3 percent from $194,510 to $190,000 and the FTEs were reduced from 0.80 to 0.70. For the National Transportation Statistics, contract costs were reduced from $204,000 to $200,000 and the FTEs were reduced from 0.80 to 0.50 resulting in a total cost reduction of 12.2 percent. International Freight data acquisition costs from the Census Bureau have been reduced from $192,728 in FY 2004 to $187,198 in FY 2005, a reduction of 2.9 percent.

Evidence: 1, The BTS Strategic Plan presents the strategic goals and the measures used to support the strategic goals. The BTS Strategic Plan, 2000-2005 is posted on the BTS website at http://www.bts.gov/publications/strategic_plan/entire.pdf. The BTS Strategic Plan, 2003-2008 is posted on the BTS website at http://www.bts.gov/publications/strategic_plan/2003_2008/pdf/entire.pdf. 2. The BTS Performance Report contains the measures, targets, timeframes, and actual performance data for each of the strategic goals. The BTS Performance Report, 2005 is posted on the BTS website at http://www.bts.gov/publications/performance_report/.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: While BTS is unique in its role in transportation statistics, it can be compared to other Federal statistical agencies that use performance measures that are consistent with the recommendations of the Interagency Council on Statistical Policy. In some areas, BTS performance on these indicators compares favorably with that of other Federal agencies. For timeliness, BTS has maintained 100% on-time data release schedules since March 2005 for all Airline information data products. This is comparable to the release schedules of the larger federal statistical agencies such as Census, Bureau of Labor Statistics, and Bureau of Economic Analysis. In addition, DOT relies on BTS to validate the quality of the data and the source and accuracy statements published in the annual Performance and Accountability Report. George Mason University evaluates the quality of report and it received the best score for data validation due to the extensive detail on the completeness and reliability of data for each performance measure. Only two other agencies received rating at that level.

Evidence: 1. Selected Regulations Related to the Office of Airline Information (14 CFR Parts 200, 217, 234, 241, 247, 248, 249, 250, 291, 298, 374a, 385, and 399) http://www.bts.gov/laws_and_regulations/docs/oai_regs.htm. 2. 7th Annual Performance Report Scorecard, Mercatus Center, George Mason University http://www.mercatus.org/repository/docLib/20060508_MC_GAP_ScorecardFY2005.pdf 3. Email from David Smallen, RITA 6/15/2006 regarding airline information data releases.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations of BTS programs have confirmed that the program is effective and achieve results. The independent National Research Council conducted a review of high quality and scope that examined the organizational structure and performance of BTS. The review reported: "During its short span of existence (since 1992), BTS has accomplished a great deal. It has issued annual reports on the transportation system that have a strong analytical focus, produced compendia and guides to transportation data, sponsored major surveys on intermodal commodity flows and household travel, provided access to a wide range of transportation data on the World Wide Web and CD-ROMs, sponsored symposia and conferences, worked with other federal agencies to standardize geographic information systems, and initiated work with the Bureau of Economic Analysis to develop a fuller accounting of the transportation sector of the U.S. economy." The Transportation Research Board has conducted peer reviews of several BTS programs and noted that the programs were effectively meeting the needs for which they were designed.

Evidence: 1. National Research Council report, The Bureau of Transportation Statistics: Priorities for the Future, National Academy Press, 1997 http://www.bts.gov/publications/bts_priorities_for_the_future/pdf/entire.pdf 2. TRB Special Report 277, Measuring Personal Travel and Goods Movements: A Review of the Bureau of Transportation Statistics' Surveys (2003) http://trb.org/publications/sr/sr277.pdf. 3. TRB Special Report 276, A Concept for a National Freight Data Program (2003) http://trb.org/publications/sr/sr276.pdf.