U.S.
DEPARTMENT OF
ENERGY

For more information about the Office of Science, go to Office of Science

Program Announcement
To DOE National Laboratories
LAB 04-03

High-Performance Network Research:
Scientific Discovery through
Advanced Computing (SciDAC)
and
Mathematical, Informational, and Computational Sciences (MICS)

SUMMARY: The Office of Advanced Scientific Computing Research (OASCR) of the Office of Science (SC), in the U.S. Department of Energy (DOE), hereby announces its interest in receiving proposals for projects in the high-performance network research program. Opportunities exist for research with a primary focus on integrated experimental networks to support high-impact science applications in the Scientific Discovery through Advanced Computing (SciDAC) program and for ultra high-speed network technologies under the Mathematical, Computational, and Information Sciences (MICS) Division. More specific information on this Announcement is outlined in the Supplementary Information section below.

DATES: All pre-proposals, referencing Program Announcement LAB 04-03, should be received by December 15, 2003 4:30 p.m., E.S.T. A response to the pre-proposals encouraging or discouraging a formal proposal generally will be communicated to the principal investigator within 14 days of receipt.

Formal proposals in response to this announcement should be received by February 25, 2004, 4:30 p.m., E.S.T., in order to be accepted for merit review and to permit timely consideration for award in Fiscal Year 2004.

ADDRESSES: All pre-proposals referencing Program Announcement LAB 04-03, should be sent via Email to Dr. Thomas D. Ndousse using the following Email address: tndousse@er.doe.gov.

Formal proposals, referencing Program Announcement LAB 04-03, must be by hardcopy, a printed original of the proposal and seven copies sent to: Dr. Thomas D. Ndousse, U.S. Department of Energy, Mathematical, Information, and Computational Sciences Division, Germantown Bldg/SC-31, Office of Science, 1000 Independence Avenue, SW, Washington DC 20585-1290. ATTN: Program Announcement LAB 04-03.

When submitting proposals by U.S. Postal Service Express Mail, any commercial mail delivery service, or when hand carried by the proposer, the following address must be used: U.S. Department of Energy, Office of Science, Mathematical, Information, and Computational Sciences Division, SC-31, 19901 Germantown Road, Germantown, MD 20874-1290, ATTN: Program Announcement LAB 04-03.

SUPPLEMENTARY INFORMATION: Emerging large-scale science projects in many areas of science such as high-energy physics, nuclear physics, climate modeling, biological sciences, etc., are anticipated to generate up to several Petabytes of data that will be transferred to geographically distant terascale computing facilities for analysis. The problems of efficient transfer of Petabyte-scale data, remote visualization of the resulting analysis, remote access to complex scientific instruments, and efficient large-scale scientific collaboration over today’s networks all present serious technical challenges to networking and science communities. Addressing these challenges calls for a new generation of highly scalable transport mechanisms that can deliver and sustain multi-Gbps to high-end scientific applications, agile networking technologies that will make bandwidth on-demand possible, innovative scalable cyber security systems that operate efficiently and effectively at ultra high-speed (10 Gbps and beyond), and intelligent network services that enable scientists to use network infrastructure with ease. These components are the critical building blocks of a new generation of ultra high-speed networks for DOE high-impact science applications.

The design of ultra high-speed networks that are effectively coupled to distributed, high-impact science applications is especially challenging because existing, widely–deployed low-speed speed network technologies do not perform well at ultra high-speeds. For example, transport protocols such as the TCP and UDP stacks, intrusion detection systems, network interface cards, network measurement tools, firewalls, and the related middleware are known to perform poorly at ultra high-speed.

Research is needed to enhance the performance of existing components and in some cases to develop radically new components that work effectively and efficiently at ultra high-speed. In addition, understanding how these components can be integrated to develop production-quality, ultra high-speed networks that can deliver end-to-end multi-Gigabits/sec to distributed scientific applications are of significant importance.

These challenges will be addressed through an integrated program that emphasizes fundamental research and experimental network engineering activities designed to demonstrate the capabilities of ultra high-speed networks under realistic high-end computing scenarios for accelerated scientific discoveries. The integrated experimental network pilots will be supported under the SciDAC program while the fundamental networking research and development will be supported under the MICS program. More information on DOE networking requirements for distributed high-end applications can be found in the following workshop reports:

(1) DOE Science Networking Challenges Workshop: Roadmap to 2008: http://www.es.net/hypertext/welcome/pr/Roadmap/index.html

(2) Office of Science High-Performance Networking Planning Workshop, http://doecollaboratory.pnl.gov/meetings/hpnpw/index.html,

(3) Ultra High-Speed Transport Protocols and Network Provisioning Workshop: http://www.csm.ornl.gov/ghpn/wk2003

A. SciDAC Program: Integrated Experimental Ultra High-Speed Networks

Background

Beyond the scientific computing and computational science research embedded in the DOE core research programs, the Office of Science invests in a portfolio of coordinated research efforts directed at exploiting the emerging capabilities of terascale and petascale computing under the collective title of Scientific Discovery through Advanced Computing (SciDAC). The research projects in the SciDAC portfolio respond to the extraordinary difficulties of realizing sustained peak performance for those scientific applications that require terascale and petascale capabilities to accomplish their research goals. In recognition of these difficulties, the SciDAC research projects are collaborative efforts involving teams of physical scientists, mathematicians, computer scientists, and computational scientists working on major software and algorithm development for problems in the core research programs of the Office of Science. Research funded in the SciDAC portfolio must address the interdisciplinary problems inherent in ultra- scale computing, problems that cannot be addressed by a single investigator or small group of investigators.

This element in high performance networking, focuses on using the science applications in the SciDAC portfolio to test and validate the capabilities of ultra high-speed networks. This effort is designed to determine and demonstrate how ultra high-speed networks, high-performance middleware, and high-end science applications can be seamlessly integrated to build a new generation network environment for accelerating scientific discoveries. All proposals submitted under this element must have three distinct but integrated components: the DOE Science UltraNet test and/or the Energy Science Network (ESnet), a set of distributed high-end science SciDAC application prototypes, and a suite of high-performance middleware tools and services to efficiently couple the high-end science applications to the underlying network. In addition, projects in this effort must satisfy the following requirements:

  • They must involve the use of an ultra high-speed network supporting a science application of national and international significance related to DOE’s mission.

  • They must use distributed high-end science applications, preferably but not necessarily existing SciDAC science applications. (A complete description of the SciDAC program can be found at: http://www.osti.gov/scidac/).

  • High-performance middleware or grid technologies must be employed to couple the selected applications to the underlying high-speed network infrastructures.

It is expected that projects will use the DOE Science UltraNet Testbed or some segment of high- performance networks such as ESnet with comparable capabilities. Detailed information on the DOE Science UltraNet testbed can be obtained at: http://www.csm.ornl.gov/ultranet, and that of ESnet at: http://www.es.net.

Specific network capabilities to be demonstrated in these integrated experimental network projects may include but are not limited to the following:

  • Petabyte-scale data distribution engineering – ultra high-speed data transfers over very long distances using TCP and non-TCP protocols, SANs or over wide-area networks, network data caching, and dynamic network provisioning network technology for on- demand data transfers, etc. This effort must include appropriate high-impact science applications with significant needs for very high-speed data transfers.

  • Scalable Network monitoring infrastructure – a collection of scalable and secure network monitoring platforms, strategically located at participating science sites and in peering points. This infrastructure must enable national and international researchers to monitor the end-to-end performance of networks, diagnose faults, and predict network performance at various layers of abstraction including the application layer. The target network environment for this infrastructure should be the DOE UltraNet testbed and/or a segment of the Energy Sciences Network (ESnet) which operates at 10 Gbps and above over multiple domains.

  • Cyber Security Infrastructure for open science Communities – a comprehensive cyber security infrastructure for a community of scientists that will enable them to collaborate and share distributed resources securely. The target science community must have well-defined shared resources and a collection of appropriate middleware services and policies to share them.

It is recommended that target science applications and tools selected for the above project be selected from previously funded SciDAC projects or projects that are consistent with its vision. A complete list of funded SciDAC projects can be found at: http://www.osti.gov/scidac/projects.

B. MICS – Base Program: Ultra High-Speed Network Engineering

The MICS aspect of this solicitation deals with research and development of ultra high-speed network technologies on a longer time horizon. It focuses primarily on deployable network transport protocols, advanced end-to-end network services, network-aware middleware, and end- to-end dynamics provisioning technologies, all of which must operate efficiently at ultra high- speed (10 Gbps and beyond). The specific technologies of current interest include but are not limited to the following:

    Ultra high-speed transport protocols – scalable transport protocol stacks that deliver and sustain multi-Gigabits/second to high-end applications efficiently on dedicated or shared single/multiple ultra high-speed channels. Such protocols could involve the extension of the existing TCP stacks or radical new non-TCP/IP approaches that could interoperate with existing network infrastructures.

    Dynamic provisioning technologies – agile network technologies to provide on-demand optical channels, wavelength scheduling, wavelength sharing, coarse-grain QOS to diverse science communities. In addition, such technologies must provide the capability to establish packet-switched, circuit-switched, or hybrid optical paths dynamically from a pool of wavelengths.

    Ultra high-speed cyber security systems – scalable cyber security systems such as firewalls, intrusion detection systems, authentication/authorizations systems, and related services that operate efficiently at ultra high-speed.

    Ultra high-speed network measurement and analysis – efficient tools and techniques for diagnosing, end-to-end performance prediction of ultra high-speed network.

Respondents are encouraged to refer to the final report of the DOE Science Networking Challenge: Roadmap to 2008 found at: http://www.osti.gov/scidac/projects.html for additional information on the Office of Science networking requirements.

Pre-proposals

The pre-proposals should consist of two to three pages of narrative describing the research objectives and technical approaches. Pre-proposals will be reviewed relative to the scope and research needs of the ASCR ultra high-speed networks for high-end scientific computing, as outlined in the summary paragraph and in the SUPPLEMENTARY INFORMATION. The pre-proposal should identify, on the cover sheet, the title of the project, the institution, principal investigator name, telephone, fax, and e-mail address. The focus element (SciDAC or MICS) for the pre-proposal should also be clearly identified.

Collaboration

Researchers are encouraged to collaborate with researchers in other institutions, such as universities, industry, non-profit organizations, federal laboratories and Federally Funded Research and Development Centers (FFRDCs), including the DOE National Laboratories, where appropriate, and to include cost sharing wherever feasible. Additional information on collaboration is available in the Application Guide for the Office of Science Financial Assistance Program that is available via the Internet at: http://www.sc.doe.gov/production/grants/Colab.html.

Program Funding

It is anticipated that up to $5 million will be available annually for awards in SciDAC and MICS Programs; up to six to ten awards are anticipated, contingent on availability of appropriated funds in Fiscal Year 2004, and the size of the awards. Multiple year funding is expected, also contingent on availability of funds and progress of the research.

Awards are expected to be at most $1.2 million per year for experimental ultra high-speed network research projects. Awards for integrated experimental ultra high-speed networks research projects are expected to be at most $1.2 million per year. Since integrated experimental networking projects are expected to be multi-institution and multi-disciplinary projects, awards under this notice would range from $150,000 to $500,000 for participation in an experimental networks project per participating project. Awards for ultra high-speed networking engineering will range from $150,000 to $300,000 per year for each single investigator. The funding period for all projects will range from two to three years subject to availability of funds. Grant applications funded under these programs will be handled as cooperative agreements.

Submission Information

The Project Description must be 20 pages or less, exclusive of attachments. It must contain an abstract or project summary on a separate page with the name of the principal investigator, mailing address, phone, FAX, and email listed. The proposal must include letters of intent from collaborators (briefly describing the intended contribution of each to the research), and short curriculum vitaes for the principal investigator and any co-PIs.

The instructions and format described below should be followed. Reference Program Announcement LAB 04-03 on all submissions and inquiries about this program.

OFFICE OF SCIENCE
GUIDE FOR PREPARATION OF SCIENTIFIC/TECHNICAL PROPOSALS
TO BE SUBMITTED BY NATIONAL LABORATORIES

Proposals from National Laboratories submitted to the Office of Science (SC) as a result of this program announcement will follow the Department of Energy Field Work Proposal process with additional information requested to allow for scientific/technical merit review. The following guidelines for content and format are intended to facilitate an understanding of the requirements necessary for SC to conduct a merit review of a proposal. Please follow the guidelines carefully, as deviations could be cause for declination of a proposal without merit review.

1. Evaluation Criteria

Proposals will be subjected to formal merit review (peer review) and will be evaluated against the following criteria which are listed in descending order of importance:

    Scientific and/or technical merit of the project

    Appropriateness of the proposed method or approach

    Competency of the personnel and adequacy of the proposed resources

    Reasonableness and appropriateness of the proposed budget

The evaluation under item 1, Scientific and/or Technical Merit of the Project, will also consider the following elements:

    a) The potential of the proposed project to make a significant impact to distributed Petabytes- scale distributed data archives and other high-end science applications.

    b) The extent to which the results of the project are extensible operational production high- performance networks, such as ESnet.

    c) The degree ultra high-speed networking technologies can inter-operate with existing networking technologies.

The evaluation under item 2, Appropriateness of the Proposed Method or Approach, will also consider the following elements:

    a) The degree to which the project adheres to the management philosophy of incorporating science applications into the project execution.

    b) The quality of the plan for ensuring interoperability and integration with related network environment software produced by other MICS and SciDAC efforts.

    c) The extent to which the project incorporates broad community (industry/academia/other federal programs) interaction.

    d) Quality and clarity of proposed work schedule and deliverables.

    e) Use of recent advances in optical network technologies, such as GMPLS to support distributed high-end applications.

The evaluation will include program policy factors such as the relevance of the proposed research to the terms of the announcement, the uniqueness of the proposer's capabilities, and demonstrated usefulness of the research for proposals in other DOE Program Offices as evidenced by a history of programmatic support directly related to the proposed work.

2. Summary of Proposal Contents

    Field Work Proposal (FWP) Format (Reference DOE Order 5700.7C) (DOE ONLY)
    Proposal Cover Page
    Table of Contents
    Abstract
    Narrative
    Literature Cited
    Budget and Budget Explanation
    Other support of investigators
    Biographical Sketches
    Description of facilities and resources
    Appendix

2.1 Number of Copies to Submit

An original and seven copies of the formal proposal/FWP must be submitted.

3. Detailed Contents of the Proposal

Proposals must be readily legible, when photocopied, and must conform to the following three requirements: the height of the letters must be no smaller than 10 point with at least 2 points of spacing between lines (leading); the type density must average no more than 17 characters per inch; the margins must be at least one-half inch on all sides. Figures, charts, tables, figure legends, etc., may include type smaller than these requirements so long as they are still fully legible.

3.1 Field Work Proposal Format (Reference DOE Order 5700.7C) (DOE ONLY)

The Field Work Proposal (FWP) is to be prepared and submitted consistent with policies of the investigator's laboratory and the local DOE Operations Office. Additional information is also requested to allow for scientific/technical merit review.

Laboratories may submit proposals directly to the SC Program office listed above. A copy should also be provided to the appropriate DOE operations office.

3.2 Proposal Cover Page

The following proposal cover page information may be placed on plain paper. No form is required.

    Title of proposed project
    SC Program announcement title
    Name of laboratory
    Name of principal investigator (PI)
    Position title of PI
    Mailing address of PI
    Telephone of PI
    Fax number of PI
    Electronic mail address of PI
    Name of official signing for laboratory*
    Title of official
    Fax number of official
    Telephone of official
    Electronic mail address of official
    Requested funding for each year; total request
    Use of human subjects in proposed project:
      If activities involving human subjects are not planned at any time during the proposed project period, state "No"; otherwise state "Yes", provide the IRB Approval date and Assurance of Compliance Number and include all necessary information with the proposal should human subjects be involved.
    Use of vertebrate animals in proposed project:
      If activities involving vertebrate animals are not planned at any time during this project, state "No"; otherwise state "Yes" and provide the IACUC Approval date and Animal Welfare Assurance number from NIH and include all necessary information with the proposal.
    Signature of PI, date of signature
    Signature of official, date of signature*

    *The signature certifies that personnel and facilities are available as stated in the proposal, if the project is funded.

3.3 Table of Contents

Provide the initial page number for each of the sections of the proposal. Number pages consecutively at the bottom of each page throughout the proposal. Start each major section at the top of a new page. Do not use unnumbered pages and do not use suffices, such as 5a, 5b.

3.4 Abstract

Provide an abstract of no more than 250 words. Give the broad, long-term objectives and what the specific research proposed is intended to accomplish. State the hypotheses to be tested. Indicate how the proposed research addresses the SC scientific/technical area specifically described in this announcement.

3.5 Narrative

The narrative comprises the research plan for the project and is limited to 5 pages per task. It should contain the following subsections:

Background and Significance: Briefly sketch the background leading to the present proposal, critically evaluate existing knowledge, and specifically identify the gaps which the project is intended to fill. State concisely the importance of the research described in the proposal. Explain the relevance of the project to the research needs identified by the Office of Science. Include references to relevant published literature, both to work of the investigators and to work done by other researchers.

Preliminary Studies: Use this section to provide an account of any preliminary studies that may be pertinent to the proposal. Include any other information that will help to establish the experience and competence of the investigators to pursue the proposed project. References to appropriate publications and manuscripts submitted or accepted for publication may be included.

Research Design and Methods: Describe the research design and the procedures to be used to accomplish the specific aims of the project. Describe new techniques and methodologies and explain the advantages over existing techniques and methodologies. As part of this section, provide a tentative sequence or timetable for the project.

Subcontract or Consortium Arrangements: If any portion of the project described under "Research Design and Methods" is to be done in collaboration with another institution, provide information on the institution and why it is to do the specific component of the project. Further information on any such arrangements is to be given in the sections "Budget and Budget Explanation", "Biographical Sketches", and "Description of Facilities and Resources".

3.6 Literature Cited

List all references cited in the narrative. Limit citations to current literature relevant to the proposed research. Information about each reference should be sufficient for it to be located by a reviewer of the proposal.

3.7 Budget and Budget Explanation

A detailed budget is required for the entire project period, which normally will be three years, and for each fiscal year. It is preferred that DOE's budget page, Form 4620.1 be used for providing budget information*. Modifications of categories are permissible to comply with institutional practices, for example with regard to overhead costs.

A written justification of each budget item is to follow the budget pages. For personnel this should take the form of a one-sentence statement of the role of the person in the project. Provide a detailed justification of the need for each item of permanent equipment. Explain each of the other direct costs in sufficient detail for reviewers to be able to judge the appropriateness of the amount requested.

Further instructions regarding the budget are given in section 4 of this guide.

* Form 4620.1 is available at web site: http://www.sc.doe.gov/production/grants/Forms-E.html

3.8 Other Support of Investigators

Other support is defined as all financial resources, whether Federal, non-Federal, commercial or institutional, available in direct support of an individual's research endeavors. Information on active and pending other support is required for all senior personnel, including investigators at collaborating institutions to be funded by a subcontract. For each item of other support, give the organization or agency, inclusive dates of the project or proposed project, annual funding, and level of effort devoted to the project.

3.9 Biographical Sketches

This information is required for senior personnel at the laboratory submitting the proposal and at all subcontracting institutions. The biographical sketch is limited to a maximum of two pages for each investigator.

3.10 Description of Facilities and Resources

Describe briefly the facilities to be used for the conduct of the proposed research. Indicate the performance sites and describe pertinent capabilities, including support facilities (such as machine shops) that will be used during the project. List the most important equipment items already available for the project and their pertinent capabilities. Include this information for each subcontracting institution, if any.

3.11 Appendix

Include collated sets of all appendix materials with each copy of the proposal. Do not use the appendix to circumvent the page limitations of the proposal. Information should be included that may not be easily accessible to a reviewer.

Reviewers are not required to consider information in the Appendix, only that in the body of the proposal. Reviewers may not have time to read extensive appendix materials with the same care as they will read the proposal proper.

The appendix may contain the following items: up to five publications, manuscripts (accepted for publication), abstracts, patents, or other printed materials directly relevant to this project, but not generally available to the scientific community; and letters from investigators at other institutions stating their agreement to participate in the project (do not include letters of endorsement of the project).

4. Detailed Instructions for the Budget
(DOE Form 4620.1 "Budget Page" may be used)

4.1 Salaries and Wages

List the names of the principal investigator and other key personnel and the estimated number of person-months for which DOE funding is requested. Proposers should list the number of postdoctoral associates and other professional positions included in the proposal and indicate the number of full-time-equivalent (FTE) person-months and rate of pay (hourly, monthly or annually). For graduate and undergraduate students and all other personnel categories such as secretarial, clerical, technical, etc., show the total number of people needed in each job title and total salaries needed. Salaries requested must be consistent with the institution's regular practices. The budget explanation should define concisely the role of each position in the overall project.

4.2 Equipment

DOE defines equipment as "an item of tangible personal property that has a useful life of more than two years and an acquisition cost of $25,000 or more." Special purpose equipment means equipment which is used only for research, scientific or other technical activities. Items of needed equipment should be individually listed by description and estimated cost, including tax, and adequately justified. Allowable items ordinarily will be limited to scientific equipment that is not already available for the conduct of the work. General purpose office equipment normally will not be considered eligible for support.

4.3 Domestic Travel

The type and extent of travel and its relation to the research should be specified. Funds may be requested for attendance at meetings and conferences, other travel associated with the work and subsistence. In order to qualify for support, attendance at meetings or conferences must enhance the investigator's capability to perform the research, plan extensions of it, or disseminate its results. Consultant's travel costs also may be requested.

4.4 Foreign Travel

Foreign travel is any travel outside Canada and the United States and its territories and possessions. Foreign travel may be approved only if it is directly related to project objectives.

4.5 Other Direct Costs

The budget should itemize other anticipated direct costs not included under the headings above, including materials and supplies, publication costs, computer services, and consultant services (which are discussed below). Other examples are: aircraft rental, space rental at research establishments away from the institution, minor building alterations, service charges, and fabrication of equipment or systems not available off- the-shelf. Reference books and periodicals may be charged to the project only if they are specifically related to the research.

a. Materials and Supplies

The budget should indicate in general terms the type of required expendable materials and supplies with their estimated costs. The breakdown should be more detailed when the cost is substantial.

b. Publication Costs/Page Charges

The budget may request funds for the costs of preparing and publishing the results of research, including costs of reports, reprints page charges, or other journal costs (except costs for prior or early publication), and necessary illustrations.

c. Consultant Services

Anticipated consultant services should be justified and information furnished on each individual's expertise, primary organizational affiliation, daily compensation rate and number of days expected service. Consultant's travel costs should be listed separately under travel in the budget.

d. Computer Services

The cost of computer services, including computer-based retrieval of scientific and technical information, may be requested. A justification based on the established computer service rates should be included.

e. Subcontracts

Subcontracts should be listed so that they can be properly evaluated. There should be an anticipated cost and an explanation of that cost for each subcontract. The total amount of each subcontract should also appear as a budget item.

4.6 Indirect Costs

Explain the basis for each overhead and indirect cost. Include the current rates.