Detailed Information on the
Polar Research Tools, Facilities and Logistics Assessment

Implement the Corrective Action Plan developed in response to the NSF IG finding on Contract Monitoring.

An external committee of visitors is completing targeted reviews of the program.

The program will improve performance targets and will continue to improve monitoring of performance against those targets.

The program should further promote its use of earned value management to promote efficiency in facilities construction.

The Budget provides funding to continue this program??s current effectiveness in supporting Arctic and Antarctic research.

The Budget includes a transfer of funding for three polar icebreakers from the U.S. Coast Guard to NSF, the primary customer for polar icebreaking services.

The program should monitor ongoing and new construction and operations to ensure efficiency, an ongoing challenge in the harsh Antarctic climate.

The agency should work with the Coast Guard and other stakeholders to ensure continued access to Antarctic research stations despite challenging ice conditions and an evolving icebreaker fleet.

The program will ensure increased timeliness of yearly project reports from investigators.

Measure: Qualitative assessment by external experts that the Polar Tools, Facilities and Logistics program has an appropriate balance between research support and investment on infrastructure improvements.

Explanation:This is a measure of decision making of the program and the ability to meet national needs for research in the polar regions.

Measure: Qualitative assessment by external experts that the Polar Tools, Facilities and Logistics program provides appropriate logistics, facilities, and science support to meet science community needs.

Explanation:This is a measure of the extent to which the program meets researcher requirements.

Measure: Qualitative assessment by external experts that the Polar Tools, Facilities and Logistics program provides appropriate protection for the health, safety, and welfare of polar program participants.

Explanation:This is a measure of the allocation of resources to ensure health and safety of researchers, contractors, and other USAP participants while they are in Antarctica and the Arctic.

Measure: Qualitative assessment by external experts that the Polar Tools, Facilities, and Logistics program effectively addresses other mission agencies' needs in their operations in the Arctic and the Antarctic.

Explanation:The program has national responsibilities with regard to any U.S. research undertaken in Antarctic. This is a measure of how well the program responds to and coordinates with other agencies on research in both polar regions.

Measure: Percent of person-days planned for Antarctic research for which the program is able to provide the necessary research support

Explanation:This accounts for all days any researcher is in Antarctica (or on the research vessels). Lost time due to items outside of the program's control (e.g. severe weather) is excluded. This measure is a proxy for efficiency and compares results to original estimates.

Measure: Percent of construction cost and schedule variances of major projects as monitored by Earned Value Management.

Explanation:This is a measure against planned cost and schedule for construction projects with a total project cost of at least $5M. The result is an average of cost and schedule variances.

Is the program purpose clear?

Explanation: The purpose of the Polar Tools, Facilities and Logistics program is to support world-class science at both Poles. This support is provided through objectives for transportation, communications and infrastructure. The program includes logistics, facilities construction, operations, and research support for research in the Antarctic and the Arctic. Research support in the Arctic is provided either through centralized support services or through competitive grants. In some cases the grants are handled through the program being reviewed here (e.g., Long Term Observations in the Arctic) or are selected competitively by merit review through other NSF programs.

Evidence: Evidence to explain the purpose of the Polar Tools, Facilities and Logistics program can be found in several documents: the NSF Strategic Plan FY 2003-2008 (www.nsf.gov/od/gpra/Strategic_Plan/FY2003-2008.pdf); National Science Foundation Act of 1950, 42 USC 1861 et. seq.; Presidential Memorandum 6646; Antarctic Treaty; Arctic Research and Policy Act; National Security Directives 71 and 318; Presidential Decision Directive 26; National Science and Technology Council report on U.S. Antarctic Program (1996); The U.S. in Antarctica: Report of the U.S. Antarctic Program External Panel (1997); U.S. Antarctic Program: Summary and Background (www.nsf.gov/od/opp/antarct/treaty/opp04001/nsf04013_full.htm).

Does the program address a specific and existing problem, interest or need?

Explanation: The United States is a leading nation in polar science, and research results have global significance. This program provides scientists in the Arctic and Antarctica with logistics and operational support as well as laboratory and infrastructure support. The program responds to National policy to have a U.S. presence in Antarctica, manifested by three year-round research stations. The program also responds to the need for improved logistical coordination and support for Arctic Research, as promulgated in the Arctic Research and Policy Act.

Evidence: Evidence is found in the NSF Strategic Plan FY 2003-2008 (www.nsf.gov/od/gpra/Strategic_Plan/FY2003-2008.pdf); National Science Foundation Act of 1950, 42 USC 1861 et. seq.; Presidential Memorandum 6646; Antarctic Treaty; Arctic Research and Policy Act; National Security Directives 71 and 318; Presidential Decision Directive 26; National Science and Technology Council report on U.S. Antarctic Program (1996); and The U.S. in Antarctica: Report of the U.S. Antarctic Program External Panel (1997).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Polar Tools, Facilities and Logistics includes logistics, operations, and research support for research in the Antarctic and the Arctic. In the Antarctic, NSF has the responsibilities as mandated in Presidential Memoranda and other documents. In the Arctic, NSF is the lead agency responsible for implementing Arctic research policy and chairs the Interagency Arctic Research Policy Committee, which is tasked with coordinating research among the twelve federal agencies supporting Arctic research and facilitating cooperation between the federal programs and other organizations. This helps ensure that efforts are not duplicative.

Evidence: Relevant information can be found in Presidential Memorandum 6646; Antarctic Treaty; Arctic Research and Policy Act; National Security Directives 71 and 318; and Presidential Decision Directive 26.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program continuously examines itself in order to provide scientists in the Arctic and Antarctic with logistics and operational support as well as laboratory and infrastructure support. Committees of Visitors regularly provide feed-back on management-related concerns and an overall assessment of the effectiveness of the program and program structure. The program also receives advice on program effectiveness and efficiency by User Committees for each Antarctic station and marine operations. User Committees meet at least once a year and are comprised of external scientists and researchers who use the facility under review. These groups provide input on whether the program is performing effectively.

Evidence: Evidence of the program's efficiency and effectiveness may be found in Committee of Visitor reports, User Committee Reports, Annual Performance Evaluation, and at the Annual Planning Conference.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Support for program beneficiaries, polar researchers, is targeted to fields of science consistent with NSF's mission. An Annual Performance Evaluation Committee reviews the prime support contractor; input is provided by 'user' organizations, grantees, and Federal operations and science managers. The program has a conference every year after the operating season with all program partners (NSF, Raytheon Polar Services Co., New York Air National Guard, USAF, Navy/SPAWAR, U.S. Coast Guard, Interior, Petroleum Helicopters Inc. and Kenn Borek Air) to review lessons learned, analyze ways to improve operations and operational efficiencies, create short- and long-term plans, and develop new initiatives. User Committees, comprised of scientists and researchers who use the facility under review, and Committees of Visitors, which review the overall program, provide a continuous feedback loop to ensure the design is appropriate.

Evidence: Evidence of Polar Tools being effectively targeted can be found in the NSF Budget Requests to Congress, User Committee Reports, and in Committee of Visitors reports.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program's long-range performance measures focus on transportation, communications and infrastructure.

Evidence: Performance measures can be found in the Measures tab.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program's long-term measures focus on providing a diversified transportation network, 24/7 communications capabilities at all operating locations, and world-class infrastructure for science and operations.

Evidence: Performance measures can be found in the Measures tab.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program annually evaluates the performance of its prime support contractor in achieving the program's long-term goals.

Evidence: Evidence is provided through the Annual Performance Evaluation Committee which reviews the prime support contractor; input is provided by 'user' organizations, grantees, Federal operations and science managers. There is an Annual Planning Conference for reviewing lessons learned, creating short- and long-term plans, and developing new initiatives.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program conducts an annual review of the performance of its prime support contractor in achieving annual goals. Goals are reviewed each year and revised to ensure they address emerging issues and challenges. Even when goals remain the same, the target performance may be increased.

Evidence: Evidence of baselines and ambitious targets is determined by the Annual Performance Evaluation Committee.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Partners include contractors, other federal agencies, and other awardees. Contractors are bound by the terms of their contracts and are evaluated by a Performance Evaluation Committee consisting of NSF personnel; by Committees of Visitors; and by user groups. Other agencies and other awardees providing support to the program commit to program goals through Memoranda of Understanding/Agreement. These participants are also evaluated by Committees of Visitors and through the use of performance measures used to target areas for improvement.

Evidence: Evidence of partners' commitments to the long-term goals of Polar Tools can be found in end of season reports, in the Annual Performance Evaluation of support contractor by NSF staff, and by external Committees of Visitors reports.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Program is evaluated every three years by an external Committee of Visitors. NSF responds to recommendations in writing. Support contractors are evaluated annually to determine their performance incentive awards. In addition, other ad hoc evaluations are conducted by the NSF Inspector General, and workshops are conducted (e.g., workshop on Antarctic communications). User Committeees comprised of external scientists and researchers who use the facilities meet at least once a year. NOTE: The weight of this question has been doubled to reflect the relative importance of external review in verifying the relevance, quality, and performance of NSF's investments.

Evidence: Evidence of evaluations being of sufficient scope and quality can be found in the Committee of Visitors Report, Inspector General reports, and User Committee reports. In addition, Memoranda of Agreement with several Federal agencies that provide expertise in several areas of importance to the program; their services include consulting, oversight, special studies.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Performance information is used to make informed decisions and is incorporated into Polar programs budget requests to the Congress. Performance information is gathered through various means (such as Performance Evaluation Committee, Committees of Visitors, user groups), and decisions regarding funding priorities are made based on this information. This process ensures alignment between performance funding priorities, and that management decisions are based on performance. Significant priority areas are highlighted in Polar programs' budget requests to the Congress. The NSF FY 2005 Budget Request to Congress was built around major goals for Polar Tools, Facilities and Logistics and clearly presents the resource requests and outlines the activities that will be supported with the funds.

Evidence: The FY 2005 NSF Budget Request to Congress, as well as previous budget requests, indicate the long-term goals of the Polar Tools program and presents the resources needed in a complete and transparent manner (pages 325-341, www.nsf.gov/bfa/bud/fy2005/pdf/fy2005.pdf).

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program has no strategic planning deficiencies, and a process is in place for long-term strategic planning for the program. No long-term strategic deficiencies have been identified by Committees of Visitors, the Inspector General, or other external groups.

Evidence: Evidence demonstrating that the Polar Tools program has no strategic planning deficiencies can be found in the McMurdo Long-Range Development Plan, the National Science Technology Council report on U.S. Antarctic Program (1996) and The U.S. in Antarctica: Report of the U.S. Antarctic Program External Panel (1997).

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals and used the results to guide the resulting activity?

Explanation: The program conducts analyses of alternatives, such as lease-or-buy or competitive selection of contractors, for major expenditures. Examples include the decision to lease the RV Gould and RV Palmer; whether to commercialize LC-130 operations; and use of different aircraft (Twin Otters and the Basler). The program has also been reviewed by external groups (e.g., Augustine Panel in 1997) to assess management decisions on alternatives.

Evidence: Evidence of meaningful and credible analyses for the Polar Tools program may be found in quarterly audits attended by other federal agencies (e.g., U.S. Army Cold Regions Research and Engineering Lab, U.S. Navy SPAWAR (meteorological and air traffic support services), U.S. Navy PACDIV (engineering design and inspection)). Other evidence may be found in ad hoc program reports and in The U.S. in Antarctica: Report of the U.S. Antarctic Program External Panel (1987).

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: The program is unique in its setting, but not in its operations. Therefore, the program is able to use competition as a method to compare the cost, schedule and performance of several potential service providers. Where competition is not used, recognized experts in federal agencies are retained through memoranda of understanding, and they use their knowledge, expertise, and skills to compare the program's operation with similar operations. Within the program, the Antarctic and Arctic activities have similar goals (i.e., supporting research in polar regions), and assess and compare where efficiencies can be gained by consolidating resources (e.g., contracting for Twin Otter services, Ice Core Drilling services, UV Monitoring, and UNAVCO (satellite use)).

Evidence: The Prime Support Contract is competed every ten years; contracted fixed- and rotary-wing aircraft contractors are competed every five years. Additional evidence may be found in Memoranda of Understanding with other Federal agencies.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: Established priorities are the basis for building the budget request that is forwarded through NSF to OMB and the Congress. Priorities are structured around four overarching areas: science support, infrastructure, transportation/logistics, and communications. Outside groups influence the process of priority setting through workshops, the Committee of Visitors, and User Committees. As an operational program, priorities may change depending on unforeseen circumstances. For example, difficult ice conditions over the past several years made it clear that providing fuel storage for at least two years at McMurdo Station should be a priority for safety of operations.

Evidence: Relevant information regarding the prioritization process for Polar Tools may be found in Annual Program Plans and the McMurdo Long-Range Development Plan. Various long-range plans focusing on transportation and communications issues.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Annual Performance Evaluation Committee reviews the prime support contractor; input is provided by 'user' organizations, grantees, and federal operations and science managers. The program has a conference every year after the operating season with all program partners (NSF, Raytheon Polar Services Co., New York Air National Guard, USAF, Navy/SPAWAR, U.S. Coast Guard, Interior, Petroleum Helicopters Inc. and Kenn Borek Air) to review lessons learned, analyze ways to improve operations and operational efficiencies, plan for the short-and long-term, and develop new initiatives. The program evaluates cost, schedule, risk, and performance goals for the South Pole Station Modernization project on a quarterly basis. A recent review focused on the trade-off between increasing the number of construction personnel deployed in order to try to reduce the planned schedule. These reviews are being instituted for all major projects funded by the program, and documentation standards are being developed that will be applied as well.

Evidence: Evidence of Polar Tools collection of performance information can be found in the Performance Evaluation Committee report and the Award Fee Board report.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The program holds the prime support contractor accountable for cost, schedule, and performance through an annual evaluation that results in either having funds withheld in the event of non-performance or having the contractor's fee reduced for poor performance. Budgets based on services to be provided are developed with other performing organizations, and budgets and performance are monitored appropriately.

Evidence: Evidence demonstrating accountability for cost, schedule and performance results can be found in the Performance Evaluation Committee report and the Award Fee Board report.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: NSF routinely obligates its funds in a timely manner. An FY 2001 study conducted by PricewaterhouseCoopers found no erroneous payments. NSF has pre-award internal controls to reduce the risk of improper payments. Funds are incrementally obligated for contracts following negotiation of annual budgets. Performance is monitored on an ongoing basis and on-site oversight is maintained during the operational season in Antarctic and by periodic site visits in the Arctic.

Evidence: Evidence of the agency's financial obligations may be found in the NSF FY 2001 Risk Assessment for Erroneous Payments, Data on NSF Carryover (found in the NSF Budget Request to Congress), Risk Assessment and Award Monitoring Guide, NSF's clean opinion on financial statements for past 6 years; the NSF Financial System, the Polar Financial Management System, and the Weekly Situation Reports from on-site NSF representatives in Antarctica.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Prime Support Contractor is competed every ten years; contracted fixed- and rotary-wing aircraft contractors are competed every five years. Logistic and operational services required have been analyzed and Memoranda of Understanding with other Federal agencies have been negotiated to obtain services to achieve program efficiencies. The Prime Support Contractor earns a fee based on performance and achievement of stated goals. Performance incentives are measured qualitatively and quantitatively (e.g., a qualitative assessment is how well the contractor integrates across the company divisions; a quantitative assessment is how much cargo is received onsite by a required date). Each year the goals set for the contractor are negotiated and revised to ensure the contractor is continually seeking improvement. The program also uses NSF GPRA goals for both operations (how many science days on ice are supported) and construction (use of Earned Value Management analysis).

Evidence: Evidence demonstrating appropriate procedures to measure and achieve efficiencies and cost effectiveness can be found in the Award Fee Board report, in individual contract documents, in contractor documentation, in the Annual Program Plan, and in GPRA Facilities and Operations reports.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Polar Tools, Facilities and Logistics program includes logistics, operations, and research support for research in the Antarctic and the Arctic. In the Antarctic, NSF has the responsibilities mandated in Presidential Memoranda and other documents, and accordingly, this program supports research conducted or sponsored by other mission agencies (e.g., NASA, NOAA, USGS). In the Arctic, NSF is the lead agency responsible for implementing Arctic research policy and chairs the Interagency Arctic Research Policy Committee, which is tasked with coordinating research among the twelve federal agencies supporting Arctic research and facilitating cooperation between the federal programs and other organizations. This helps ensure that efforts are not duplicative.

Evidence: Evidence relevant to demonstrating the Polar Tools program's coordination and collaboration with similar programs may be found in Presidential Memorandum 6646, the Antarctic Treaty, the Arctic Research and Policy Act, National Security Directives 71 and 318, Presidential Decision Directive 26, and in various memoranda of understanding.

Does the program use strong financial management practices?

Explanation: NSF uses strong financial management practices. NSF was the first federal agency to receive a 'green light' for financial management on the President's Management Agenda scorecard. NSF has received a clean opinion on its financial audits for the last six years. The NSF is committed to providing quality financial management to all its stakeholders. It honors that commitment by preparing annual financial statements in conformity with generally accepted accounting principles in the U.S. and then subjecting the statement to independent audits. Supplementary statements are also prepared including Budgetary Resources by Major Accounts, Intragovernmental Balances, Deferred Maintenance, and Stewardship Investments. In addition, monthly status and progress reports are submitted to the program by every organization funded (e.g., Raytheon Polar Services Co., New York Air National Guard, Navy/SPAWAR). The program uses sound project management techniques, including Earned Value Analysis to track progress on large projects.

Evidence: Evidence of strong financial management practices in the Polar Tools program can be found in POFMS (Polar Financial Management System) and in the monthly reports status reports.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Committees of Visitors provide feedback on management-related concerns. In addition, the Foundation conducts an annual review to assess administrative and financial systems and procedures to ensure that effective management controls are in place and that any deficiencies are identified and addressed. No management deficiencies have been identified for the program in the above reviews or in other external reviews (Augustine Report, 1997; Inspector General reports).

Evidence: Reports indicating no significant management deficiencies in the annual Performance and Accountability Reports (www.nsf.gov/pubsys/ods/getpub.cfm?par), the NSF Business Analysis, the 2001 Committee of Visitors Report for the Polar Research Support Section, Advisory Committee review of Committee of Visitors reports, the Annual report to senior management, and IG reports and NSF responses.

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: The Budget Request for the program provides clearly defined deliverables (e.g., complete the power plant at McMurdo Station by a given fiscal year). The Earned Value Management system used to manage projects provides ongoing oversight of the defined deliverables, and reporting on facilities for GPRA provides an annual assessment of progress on deliverables. On a micro level, awards made as contracts or cooperative agreements have clearly defined deliverables and cost and schedule goals.

Evidence: Documentation is contained in individual contracts and cooperative agreements.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Most of the funding under Polar Tools, Facilities and Logistics is handled by contracts. The contracts are competitively bid based on a variety of factors, including merit. Grants that are made under this program rely upon NSF's competitive, merit review process that includes external peer evaluation.

Evidence: Evidence of grants awarded through a clear competitive process may be found in the FY 2003 Report on NSF Merit Review System (www.nsf.gov/nsb/documents/2004/MRreport_2003_final.pdf). Additional information may be found in NSF's Annual Acquisition Plan which includes Polar Facilities plans (www.nsf.gov/bfa/dacs/contracts/acquisitionplan04.htm) and the annual Performance and Accountability Reports (www.nsf.gov/pubs/2004/nsf0410/start.htm). An example of an RFP can be reviewed at www1.eps.gov/spg/NSF/DCPO/CPO/DACS%2D030057/listing.html.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: In FY 2002, NSF established a formal Award Monitoring and Technical Assistance Program (AM&TAP) based on financial and administrative risk assessment of NSF awardee institutions and with a primary focus to on-site monitoring. AM&TAP is a collaborative effort between administrative and financial managers/technical staff and NSF program managers. NSF maintains scientific oversight of all awards through annual and final project reports, and funds are tracked (via reporting systems) to ensure that funds are used for their designated purpose. This program specifically evaluates cost, schedule, risk and performance goals for the South Pole Station Modernization project on a quarterly basis. These reviews are being instituted for all major projects funded by the program, and documentation standards are being developed that will be applied as well. The program deploys federal agencies as on-site representatives during the operating season (e.g., Navy/PACDIV). Contract employees also provide oversight during the entire year.

Evidence: Oversight activities which demonstrate a sufficient knowledge of grantee activities may be found in the annual, interim and final project reports; FY 2003 Report on NSF Merit Review System (www.nsf.gov/nsb/documents/2004/MRreport_2003_final.pdf); Risk Assessment and Award Monitoring Guide; clean audit opinions for past six years; President's Management Agenda Scorecard for Financial Management; South Pole Station Modernization quarterly audits; reports from Federal agencies of activities reviewed or examined; SPSM quarterly review books; and RPSC Project Procedures.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The Polar Tools, Facilities and Logistics program makes possible the research conducted by grantees/awardees in Antarctica and the Arctic. Information on the support available is posted on public websites (see Evidence). NSF Grant General Conditions require that results of NSF-supported research be published in the open literature and that NSF support is appropriately referenced/cited. NSF's annual Performance and Accountability Report and its annual Budget Request contain highlights of NSF-supported research. Principal Investigators provide annual progress reports to NSF, which are examined and approved/disapproved by the program officers.

Evidence: Information on the support available in the Arctic and Antarctic is available at: www.nsf.gov/od/opp/support/start.htm; www.polar.org/ www.nsf.gov/od/opp/arctic/suplog.htm; www.vecopolar.com/. Grantee information is provided in annual Performance and Accountability Reports (www.nsf.gov/pubs/2004/nsf0410/start.htm); NSF Summary of the FY 2005 Budget Request to Congress; Grant General Conditions (www.nsf.gov/home/grants/gc102.pdf); Budget Internet Information Site (dellweb.bfa.nsf.gov/).

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: Over three-fourths of program funds are awarded competitively. Performance management techniques are used throughout the Program. Funds are incrementally obligated following negotiation of annual budgets. Performance is monitored on an ongoing basis and on-site oversight is maintained during the operational season. Periodic reviews are performed by program personnel.

Evidence: Evidence of program quality for Polar Tools can be found in the end of season reports and in the Annual Performance Evaluation of support contractor.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has demonstrated progress in all three of its long-term performance goals -- infrastructure, communications, and transportation.

Evidence: Evidence of adequate progress in achieving Polar Tools' long-term performance goals can be found in the South Pole Overland Traverse progress and in the telemedicine capabilities at all operating stations and research vessels. In addition, the McMurdo Long-Range Development Plan was developed and several projects have already been implemented (Waste Water Treatment Plant, T-Site Replacement and Joint Space Operations Center) or are in-process (Power Plant).

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program annually sets goals for performance by the prime support contractor and by other organizations (e.g., number of flights, accuracy of weather forecasting). The program achieves its annual goals relating to infrastructure, transportation, and communications.

Evidence: Evidence of goal achievement in Polar Tools is from the Performance Evaluation Committee and in Performance Statistics for airlift providers and weather forecasters.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The recent report on Arctic research support and logistics commented, "substantial progress has been made in year-round access for researchers, in protecting health and safety, and in improving collaboration and communication between researchers and arctic residents.' Each year, the percentage of fee earned by the prime support contractor has risen, direct evidence of demonstrated improvement. The program is beginning to use earned-value management to maintain efficiency in construction of facilities. The program has also begun measuring improvement in energy use at McMurdo based operations through enhanced reliance on alternative energy sources.

Evidence: Evidence supporting Polar Tools performance is included in "Arctic Research Support and Logistics," report of the Arctic Research Consortium of the U.S. (2003) and in "The U.S. in Antarctica: Report of the U.S. Antarctic Program External Panel" (1997). Cost effectiveness can be found in the Performance Evaluation Committee report and the Award Fee Board report.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The program has a unique mission ' support of world-class science at both Poles, so comparisons are not easily made. The recent report on Arctic research support and logistics commented, "substantial progress has been made in year-round access for researchers, in protecting health and safety, and in improving collaboration and communication between researchers and arctic residents.' The 1997 External Panel report on USAP indicated "...many of the U.S. assets and programs in Antarctica are unparalleled in scope or capability."

Evidence: Evidence supporting Polar Tools performance is included in "Arctic Research Support and Logistics," report of the Arctic Research Consortium of the U.S. (2003) and in "The U.S. in Antarctica: Report of the U.S. Antarctic Program External Panel" (1997).

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Activities are reviewed by Committees of Visitors. The latest COV indicated: "performance ... is most directly judged by assessing the effectiveness of the Office of Polar Programs in accomplishing its primary mission ~ conducting science in Antarctica. While research support operations can and will be judged on metrics of efficiency, productivity, on-time performance, and cost effectiveness, it is the more intangible measures related to the quality, impact, and relevance of Antarctic science that the success of Polar Research Support Section (PRSS) should ultimately be judged against. In all measures of performance, it is the COV's assessment that Antarctic science is strong, it is relevant, it is cutting edge, and it is by far the best being performed in the world today. The Committee, therefore, concludes that PRSS is accomplishing its major objectives as reflected in the state and health of U.S. Antarctic science."

Evidence: Independent evaluations of sufficient scope and quality are performed by the Performance Evaluation Committee. Additional information can be found in Performance Statistics and in the Committee of Visitors Reports. NOTE: The weight of this question has been doubled to reflect the importance of independent evaluation in verifying relevance, quality and performance of NSF's investments.

Were program goals achieved within budgeted costs and established schedules?

Explanation: The program has an excellent track record operating within budgeted costs and established schedules. However, the program's operating locations impose obstacles that are not found in other programs, such as weather delays and recent delays caused by unusually thick ice in McMurdo Sound. In the latter case funds were reprogrammed from other accounts in NSF in FY 2003 to help defray the costs.

Evidence: Evidence is contained in end-of-season reports and monthly cost and schedule reports by WBS level which are submitted to NSF.