• A Flagship Computing Facility, the National Energy Research Scientific Computing Center (NERSC), to provide the robust, high-end computing resources needed by a broad range of scientific research programs.
• Topical Computing Facilities to provide computing resources tailored for specific scientific applications and to serve as the focal point for an application community as it strives to optimize its use of terascale computers.
• Experimental Computing Facilities to assess the promise of new computing technologies being developed by the computer industry for scientific applications.

The Benefits

The Scientific Computing Software Infrastructure, along with the upgrades to the hardware infrastructure, will enable laboratory and university researchers to solve the most challenging scientific problems faced by the Office of Science at a level of accuracy and detail never before achieved. These developments will have significant benefit to all of the government agencies who rely on high-performance scientific computing to achieve their mission goals as well as to the U.S. high-performance computing industry.

Background: Integrated Software Infrastructure Centers

This announcement addresses the Mathematical and Computing Systems Software Environment element of the SciDAC Scientific Computing Software Infrastructure. ISIC envisions a comprehensive, integrated, scalable, and robust high performance software infrastructure, which overcomes difficult technical challenges to enable the effective use of terascale systems by SciDAC applications. ISIC addresses needs for: new algorithms which scale to parallel systems having thousands of processors; methodology for achieving portability and interoperability of complex high performance scientific software packages; operating systems tools and support for the effective management of terascale and beyond systems; and effective tools for feature identification, data management and visualization of petabyte-scale scientific data sets. ISIC provides the essential computing and communications infrastructure for support of SciDAC applications. The ISIC effort encompasses a multi-discipline approach with activities in:

• Algorithms, methods, and libraries -- Algorithms, methods and libraries that are fully scalable to many thousands of processors with full performance portability.
• Program development environments and tools -- Component-based, fully integrated, terascale program development and runtime tools which scale effectively and provide maximum utility and ease-of-use to developers and scientific end users.
• Operating system software and tools -- Systems software that scales to tens of thousands of processors, supports high performance application-level communication and provides the highest levels of fault tolerance, reliability, manageability, and ease of use for system administrators, tool developers and end users.
• Visualization and data management systems -- Scalable, intuitive systems fully supportive of SciDAC application requirements for moving, storing, analyzing, querying, manipulating and visualizing multi-petabytes of scientific data and objects.

• Create mathematical and/or computing systems software to enable scientific simulation codes to take full advantage of the extraordinary capabilities of terascale computers;
• Work closely with application teams and other SciDAC teams to ensure that the most critical computer science and applied mathematics issues are addressed in a timely and comprehensive fashion; and
• Address all aspects of the successful research software lifecycle including transition of a research code into a robust production code and long term software evolution and maintenance and end user support.

This announcement is one of several that address the initial requirements of the SciDAC program. The focus is on four topics: 1) algorithms, methods and libraries; 2) program development environments and tools; 3) operating systems software and tools; and 4) visualization and data management. Responses to this announcement may propose work in one or more of these areas and may be single institution efforts or partnerships that involve many organizations. It is expected that most, if not all, of the proposed activities will be organized as ETCs. Specific areas of interest include, but are not limited to, the following examples listed for each subtopic:

1) Algorithms, methods and mathematical libraries

a) Mesh generation and discretization technology. Tools to facilitate the generation and partitioning of all types of meshes (structured, unstructured, and chimera (overlapping)) designed for many thousands of processors.

b) Mathematical analysis and scalable numerical algorithms. Mathematical methods to help SciDAC applications achieve high performance on hierarchical memory terascale computers such as multiscale analysis, multilevel methods, and fast transforms capable of spanning multiple spatial and temporal scales. Resultant algorithms must be deployed in component-based mathematical software and made available to a broad range of DOE mission areas.

a) High Performance Component Architectures. Component technology that builds upon and extends commercial component architectures to support high performance parallel components, low-latency, high bandwidth communication among components, and efficient data and work redistribution.

b) Code Design and Development Tools. Scaling methodology to deploy existing parallel code development environments on multi-teraflops SciDAC systems. Support for multi-language applications including C, C++, UPC, Fortran, Co-Array Fortran, Python and Java; parallel programming libraries such as MPI, OpenMP, thread libraries, the Global Array library; and multi-level hierarchical memory programming models.

c) Code Correctness and Validation. Debugging tools that implement emerging community standards in parallel debuggers and automated data dependency analysis. Relative debugging methodology for comparing at run time the execution of two versions of a code.

d) Performance Tools. Evaluation of existing research and commercial performance analysis tools, both tracefile-based and dynamic, for scalability and suitability for SciDAC applications. Performance metrics and benchmarks which enable reliable and credible performance predictions of application codes on terascale and larger systems. Tools which link hardware counters to meaningful terascale system performance characteristics and application performance.

a) Terascale System Resource Management. Modular infrastructure for resource management on terascale clusters including resource scheduling, meta-scheduling, node daemon support, comprehensive usage accounting and user interfaces that also emphasizes portability to terascale vendor operating systems.

b) Terascale System Support. Scalable checkpointing and improved runtime steering for early deployment. Methodology for analyzing tradeoffs between fault tolerance and peak performance. Support for robust runtime job management and I/O systems that are tolerant of component failure. Scalable tools for system administration including initial system boot, system updates, job launch and system utilities.

c) High Performance Communication. Operating system support for application level communication which scales to thousands of processors, provides minimum latency and maximum bandwidth between parallel application processes. Innovative approaches to terascale operating system architectures including non-uniform kernel support for computational, service, interactive and i/o nodes.

a) Data Management Systems. Data exchange methods and standardizations that facilitate collaborative applications. Innovative Database Management Systems (DBMS) approaches for high throughput parallel I/O and complex queries of large scientific databases. Hierarchical data storage systems involving tertiary storage media that are sequential. Agent methodology for feature extraction and complex query operations. Tools for user-driven and automatic clustering, reclustering or replication of objects to maximize retrieval efficiency. Collaborations with the DBMS and tertiary storage vendor industry are encouraged.

b) Visualization. Vector/tensor field visualization in 3-D. Modes of visualization for interpretation and understanding of large datasets. Remote and collaborative visualization methods. Characterization of simulation, experimental and visualization errors/uncertainties. Adaptive, multiresolution, parallel and scalable visualization algorithms. Innovative techniques for exploring multi-dimensional, multi-discipline data sets.

Integration of Software Components and Tools

Responses to this announcement should cover the full range of activities from basic research to development of software that can be deployed to the SciDAC applications communities. It is critical that these submissions demonstrate effective strategies for coupling with requirements from applications researchers and ensuring that software developed will interoperate with software developed by other ISIC activities and be effectively deployed to SciDAC computing facilities and applications groups.

ISIC envisions a fully integrated software environment that provides both robustness and ease of use to the end user application scientist. Implementation of this vision will be coordinated through a participatory management process with input from ISIC teams and other key participants of SciDAC. As component and tool implementations mature, each team will be expected to develop the necessary technology to fully and smoothly incorporate their software tools into the ISIC environment.

ISIC activities play a critical cross-cutting role in the SciDAC. ISIC goals require significant interactions, ranging from the joint development and deployment of tools and technologies into the applications community, to the incorporation of needed capabilities into new products and systems. ISIC researchers will need to interact closely with diverse groups including: applications scientists, vendor providers, the DOE ASCI program, and other federal agency programs addressing complementary goals. To support and facilitate the maximum impact of the SciDAC Scientific Computing Software Infrastructure, high emphasis will be placed on ensuring that source code is fully and freely available for use and modification throughout the scientific computing community.

This announcement is focused on larger ETC efforts in support of the SciDAC program. Field Work Proposals to the OASCR base program which may have the potential for contributing to the ISIC software infrastructure should so indicate.

Collaboration

Applicants are encouraged to collaborate with researchers in other institutions, such as: universities, industry, non-profit organizations, and other DOE National Laboratories, where appropriate.

Program Funding

It is anticipated that up to $11 million annually will be available for multiple awards for these components of the ISIC program. Initial awards will be made in FY 2001 in the categories described above, and proposals may request project support for up to five years. All awards are contingent on the availability of funds, research progress, and programmatic needs. Annual budgets for successful ISIC projects are expected to range from $2,000,000 to $4,000,000 per project. Annual budgets may increase in the outyears but should remain within the overall annual maximum guidance. Any proposed effort that exceeds the annual maximum in the outyears should be separately identified for potential award increases if additional funds become available.

Preproposals

Preproposals are strongly encouraged but not required prior to submission of a full proposal. However, notification of a successful preproposal is not an indication that an award will be made in response to the formal proposal. The preproposal should identify on the cover sheet the institution, Principal Investigator name(s), address(s), telephone, and fax number(s) and E-mail address(es), title of the project, and the field of scientific research. A brief (one-page) vitae should be provided for each Principal Investigator. The preproposal should consist of a two to three page narrative describing the research project objectives, the approach to be taken, and a description of any research partnerships. Preproposals will be reviewed by DOE relative to the scope and research needs of the ISIC program.

DATES: Preproposals referencing Program Announcement LAB 01-07 should be received by January 31, 2001. Formal proposals in response to this announcement should be received by 4:30 p.m., E.S.T., March 15, 2001 to be accepted for merit review and funding in FY 2001.

ADDRESSES: Preproposals referencing Program Announcement LAB01-07 should be sent to: Dr. Frederick C. Johnson, Office of Advanced Scientific Computing Research, SC-31, Office of Science, U.S. Department of Energy, 19901 Germantown Road, Germantown, MD 20874-1290.

Formal proposals referencing Program Announcement LAB01-07, should be sent to: Dr. Frederick C. Johnson, Office of Science, U.S. Department of Energy, 19901 Germantown Road, Germantown, MD 20874-1290, ATTN: Program Announcement LAB01-07. This address must be used when submitting proposals by U.S. Postal Service Express Mail or any commercial mail delivery service, or when hand-carried by the applicant. An original and seven copies of the proposal must be submitted. In addition, electronic copies in pdf file format of all proposal material are encouraged.

FOR FURTHER INFORMATION CONTACT: Dr. Frederick C. Johnson, Office of Science, U.S. Department of Energy, 19901 Germantown Road, Germantown, MD 20874-1290, telephone: (301) 903-5800, E-mail: fjohnson@er.doe.gov, fax: (301) 903-7774.

The instructions and format described below should be followed. Reference Program Announcement LAB01-07 on all submissions and inquiries about the program.

Proposals from National Laboratories submitted to the Office of Science (SC) as a result of this program announcement will follow the Department of Energy Field Work Proposal process with additional information requested to allow for scientific/technical merit review. The following guidelines for content and format are intended to facilitate an understanding of the requirements necessary for SC to conduct a merit review of a proposal. Please follow the guidelines carefully, as deviations could be cause for declination of a proposal without merit review.

1. Evaluation Criteria

Proposals will be subjected to formal merit review (peer review) and will be evaluated against the following criteria which are listed in descending order of importance:

Scientific and/or technical merit of the project
Appropriateness of the proposed method or approach
Competency of the personnel and adequacy of the proposed resources
Reasonableness and appropriateness of the proposed budget

a) The potential of the proposed project to make a significant impact in the effectiveness of SciDAC applications researchers;
b) The demonstrated capabilities of the applicants to perform basic research related to ISIC and transform these research results into software that can be widely deployed;
c) The likelihood that the algorithms, methods, mathematical libraries, and software components that result from this effort will have impact on science disciplines outside of the SciDAC applications projects;
d) Identification and approach to software integration and long term support issues, including component technology, documentation, test cases, tutorials, end user training, and quality maintenance and evolution.

a) Quality of the plan for effective coupling to applications researchers;
b) Quality of plan for ensuring interoperability and integration with software produced by other ISIC and SciDAC efforts;
c) Viability of plan for deployment of software to SciDAC facilities and applications groups;
d) Knowledge of and coupling to other efforts in high performance scientific computing software such as the DOE ACTS program, the DOE ASCI program and the NSF ITR program;
e) Quality and clarity of proposed work schedule and deliverables.

The evaluation will include program policy factors such as the relevance of the proposed research to the terms of the announcement, the uniqueness of the proposer's capabilities, and demonstrated usefulness of the research for proposals in other DOE Program Offices as evidenced by a history of programmatic support directly related to the proposed work.

2. Summary of Proposal Contents

Field Work Proposal (FWP) Format (Reference DOE Order 5700.7C)
Proposal Cover Page
Table of Contents
Abstract
Narrative
Literature Cited
Budget and Budget Explanation
Other support of investigators
Biographical Sketches
Description of facilities and resources
Appendix

An original and seven copies of the formal proposal/FWP must be submitted.

3. Detailed Contents of the Proposal

Proposals must be readily legible, when photocopied, and must conform to the following three requirements: the height of the letters must be no smaller than 10 point with at least 2 points of spacing between lines (leading); the type density must average no more than 17 characters per inch; the margins must be at least one-half inch on all sides. Figures, charts, tables, figure legends, etc., may include type smaller than these requirements so long as they are still fully legible.

3.1 Field Work Proposal Format (Reference DOE Order 5700.7C)

The Field Work Proposal (FWP) is to be prepared and submitted consistent with policies of the investigator's laboratory and the local DOE Operations Office. Additional information is also requested to allow for scientific/technical merit review. Laboratories may submit proposals directly to the SC Program office listed above. A copy should also be provided to the appropriate DOE operations office.

3.2 Proposal Cover Page

The following proposal cover page information may be placed on plain paper. No form is required.

Title of proposed project
SC Program announcement title
Name of laboratory
Name of principal investigator (PI)
Position title of PI
Mailing address of PI
Telephone of PI
Fax number of PI
Electronic mail address of PI
Name of official signing for laboratory*
Title of official
Fax number of official
Telephone of official
Electronic mail address of official
Requested funding for each year; total request
Use of human subjects in proposed project:
If activities involving human subjects are not planned at any time during the proposed project period, state "No"; otherwise state "Yes", provide the IRB Approval date and Assurance of Compliance Number and include all necessary information with the proposal should human subjects be involved.
Use of vertebrate animals in proposed project:
If activities involving vertebrate animals are not planned at any time during this project, state "No"; otherwise state "Yes" and provide the IACUC Approval date and Animal Welfare Assurance number from NIH and include all necessary information with the proposal.
Signature of PI, date of signature
Signature of official, date of signature*
*The signature certifies that personnel and facilities are available as stated in the proposal, if the project is funded.

Provide the initial page number for each of the sections of the proposal. Number pages consecutively at the bottom of each page throughout the proposal. Start each major section at the top of a new page. Do not use unnumbered pages and do not use suffices, such as 5a, 5b.

3.4 Abstract

Provide an abstract of no more than 250 words. Give the broad, long-term objectives and what the specific research proposed is intended to accomplish. State the hypotheses to be tested. Indicate how the proposed research addresses the SC scientific/technical area specifically described in this announcement.

3.5 Narrative

The narrative comprises the research plan for the project and is limited to 25 pages. It should contain the following subsections:

Background and Significance: Briefly sketch the background leading to the present proposal, critically evaluate existing knowledge, and specifically identify the gaps which the project is intended to fill. State concisely the importance of the research described in the proposal. Explain the relevance of the project to the research needs identified by the Office of Science. Include references to relevant published literature, both to work of the investigators and to work done by other researchers.

Preliminary Studies: Use this section to provide an account of any preliminary studies that may be pertinent to the proposal. Include any other information that will help to establish the experience and competence of the investigators to pursue the proposed project. References to appropriate publications and manuscripts submitted or accepted for publication may be included.

Research Design and Methods: Describe the research design and the procedures to be used to accomplish the specific aims of the project. Describe new techniques and methodologies and explain the advantages over existing techniques and methodologies. As part of this section, provide a tentative sequence or timetable for the project.

Subcontract or Consortium Arrangements: If any portion of the project described under "Research Design and Methods" is to be done in collaboration with another institution, provide information on the institution and why it is to do the specific component of the project. Further information on any such arrangements is to be given in the sections "Budget and Budget Explanation", "Biographical Sketches", and "Description of Facilities and Resources".

3.6 Literature Cited

List all references cited in the narrative. Limit citations to current literature relevant to the proposed research. Information about each reference should be sufficient for it to be located by a reviewer of the proposal.

3.7 Budget and Budget Explanation

A detailed budget is required for the entire project period, which normally will be three years, and for each fiscal year. It is preferred that DOE's budget page, Form 4620.1 be used for providing budget information*. Modifications of categories are permissible to comply with institutional practices, for example with regard to overhead costs.

A written justification of each budget item is to follow the budget pages. For personnel this should take the form of a one-sentence statement of the role of the person in the project. Provide a detailed justification of the need for each item of permanent equipment. Explain each of the other direct costs in sufficient detail for reviewers to be able to judge the appropriateness of the amount requested.

Further instructions regarding the budget are given in section 4 of this guide.
* Form 4620.1 is available at web site: http://www.sc.doe.gov/production/grants/forms.html

3.8 Other Support of Investigators

Other support is defined as all financial resources, whether Federal, non-Federal, commercial or institutional, available in direct support of an individual's research endeavors. Information on active and pending other support is required for all senior personnel, including investigators at collaborating institutions to be funded by a subcontract. For each item of other support, give the organization or agency, inclusive dates of the project or proposed project, annual funding, and level of effort devoted to the project.

3.9 Biographical Sketches

This information is required for senior personnel at the laboratory submitting the proposal and at all subcontracting institutions. The biographical sketch is limited to a maximum of two pages for each investigator.

3.10 Description of Facilities and Resources

Describe briefly the facilities to be used for the conduct of the proposed research. Indicate the performance sites and describe pertinent capabilities, including support facilities (such as machine shops) that will be used during the project. List the most important equipment items already available for the project and their pertinent capabilities. Include this information for each subcontracting institution, if any.

3.11 Appendix

Include collated sets of all appendix materials with each copy of the proposal. Do not use the appendix to circumvent the page limitations of the proposal. Information should be included that may not be easily accessible to a reviewer. Reviewers are not required to consider information in the Appendix, only that in the body of the proposal. Reviewers may not have time to read extensive appendix materials with the same care as they will read the proposal proper. The appendix may contain the following items: up to five publications, manuscripts (accepted for publication), abstracts, patents, or other printed materials directly relevant to this project, but not generally available to the scientific community; and letters from investigators at other institutions stating their agreement to participate in the project (do not include letters of endorsement of the project).

4. Detailed Instructions for the Budget
(DOE Form 4620.1 "Budget Page" may be used)

4.1 Salaries and Wages List the names of the principal investigator and other key personnel and the estimated number of person-months for which DOE funding is requested. Proposers should list the number of postdoctoral associates and other professional positions included in the proposal and indicate the number of full-time-equivalent (FTE) person-months and rate of pay (hourly, monthly or annually). For graduate and undergraduate students and all other personnel categories such as secretarial, clerical, technical, etc., show the total number of people needed in each job title and total salaries needed. Salaries requested must be consistent with the institution's regular practices. The budget explanation should define concisely the role of each position in the overall project.

4.2 Equipment

DOE defines equipment as "an item of tangible personal property that has a useful life of more than two years and an acquisition cost of $5000 or more." Special purpose equipment means equipment which is used only for research, scientific or other technical activities. Items of needed equipment should be individually listed by description and estimated cost, including tax, and adequately justified. Allowable items ordinarily will be limited to scientific equipment that is not already available for the conduct of the work. General purpose office equipment normally will not be considered eligible for support.

4.3 Domestic Travel

The type and extent of travel and its relation to the research should be specified. Funds may be requested for attendance at meetings and conferences, other travel associated with the work and subsistence. In order to qualify for support, attendance at meetings or conferences must enhance the investigator's capability to perform the research, plan extensions of it, or disseminate its results. Consultant's travel costs also may be requested.

4.4 Foreign Travel

Foreign travel is any travel outside Canada and the United States and its territories and possessions. Foreign travel may be approved only if it is directly related to project objectives.

4.5 Other Direct Costs

The budget should itemize other anticipated direct costs not included under the headings above, including materials and supplies, publication costs, computer services, and consultant services (which are discussed below). Other examples are: aircraft rental, space rental at research establishments away from the institution, minor building alterations, service charges, and fabrication of equipment or systems not available off-the-shelf. Reference books and periodicals may be charged to the project only if they are specifically related to the research.

a. Materials and Supplies

The budget should indicate in general terms the type of required expendable materials and supplies with their estimated costs. The breakdown should be more detailed when the cost is substantial.

b. Publication Costs/Page Charges

The budget may request funds for the costs of preparing and publishing the results of research, including costs of reports, reprints page charges, or other journal costs (except costs for prior or early publication), and necessary illustrations.

c. Consultant Services

Anticipated consultant services should be justified and information furnished on each individual's expertise, primary organizational affiliation, daily compensation rate and number of days expected service. Consultant's travel costs should be listed separately under travel in the budget.

d. Computer Services

The cost of computer services, including computer-based retrieval of scientific and technical information, may be requested. A justification based on the established computer service rates should be included.

e. Subcontracts

Subcontracts should be listed so that they can be properly evaluated. There should be an anticipated cost and an explanation of that cost for each subcontract. The total amount of each subcontract should also appear as a budget item.

4.6 Indirect Costs

Explain the basis for each overhead and indirect cost. Include the current rates.

______________________________
FOOTNOTES:
1) This workshop was sponsored by the National Science Foundation and the Department of Energy and hosted by the National Academy of Sciences on July 30-31, 1998. Copies of the report may be obtained from: http://www.er.doe.gov/production/octr/mics/index.html
2) Copies of the PITAC report may be obtained from: http://www.ccic.gov/ac/report/.
3) Copies of the SC computing plan, Scientific Discovery through Advanced Computing, can be downloaded from the SC website at: http://www.sc.doe.gov/production/octr/index.html.