FOR FURTHER INFORMATION CONTACT:

Dr. Frederick Johnson
Telephone: (301) 903-5800
Fax: (301) 903-7774
E-mail: fjohnson@ascr.doe.gov.

The complexity and scale of petascale systems and large application codes represent major challenges for development tools including: radical increases in node and processor core counts, support for multi-mode parallelism, reduced memory per core, heterogeneous nodes, and support for fault tolerance. Application developer and user needs for these systems include: a means for debugging at scale, increased support for memory debugging, memory characterization tools, both lightweight and heavyweight tools, performance analysis tool support for serial code segments, multithreaded segments and multimode segments, and means for understanding and optimizing for topology related performance.

The research activities supported by this activity need to bridge the gap between large complex applications and next-generation hardware, including interactions with novel architectures. Consequently, there are a wide variety of research topics that are appropriate for this effort. Example candidate topics are provided below, but research in other relevant areas and combinations of areas is encouraged.

Performance Tools

Automated Diagnosis and Remediation -- New approaches to performance optimization which move beyond manual methods and enable greater automation and which support automation of diagnosis, optimization and anomaly detection.

Load Imbalance Detection -- Highly scalable methodology for detecting load imbalance in applications running on hundreds of thousands of processor cores. Tools which provide root cause analysis in addition to detection.

Heterogeneous, Hierarchical Architecture Support -- Performance tools which support multilevel parallel paradigms, including hybrid OpenMP/MPI programs. Tools which capture and relate performance and reliability problems to source code in ways that make multilevel performance optimization possible and practical.

Correctness Tools

Scalable Debuggers -- Both lightweight and heavyweight approaches to scalable debugging that support of ease of use, error detection at scale, and in-depth root cause analyses.

Memory Usage -- Both lightweight and heavyweight tools for monitoring memory utilization (especially memory leaks and overall memory consumption) and tools to find programming errors in the way memory is accessed.

Thread Correctness -- Multi-platform tools that enable users to detect incorrect use of parallel programming techniques including thread correctness checkers and Message Passing Interface (MPI) usage checkers. Tools which assess the validity of memory references, track locks that are held when memory is accessed and verify that no potential race condition exists.

Scalable Infrastructure

Data Management and Communication -- Support for all aspects of the gathering, reduction, and storage of application information and metadata. Support for communicating information among tool components on different nodes, getting information from external sources such as the operating system, compiler, scheduler, and runtime system, and exchanging information between tools.

Scheduler and Operating System Interaction -- Support for close coordination of tools with the scheduler, e.g. for tool launch on multiple nodes, and the operating system , e.g. process control interfaces for access to thread information and low overhead access to hardware counters.

Binary Manipulation -- Support for binary analysis of optimized and stripped programs, and the ability to generate new binaries with instrumentation.

Development Environment

Application Build Tools -- Tool support for radical improvements in the management of the application build process that address the complexities arising from multiple target systems, operating systems, libraries and software versions. Also support for common option sets, command line interfaces, shared libraries and dynamic link order.

Mixed Language Environments -- Tool support for mixed language programming including traditional languages, Fortran, C, C++; scripting languages, Python; and emerging languages such as the PGAS languages UPC, Co-array Fortran and the HPCS languages.

Compiler Infrastructure -- A flexible, extendible, portable, open source compiler infrastructure to support efficient information transfer between compile time analyses and tools and runtime analyses and tools.

Program Transformations -- Tools supporting source-to-source transformations to enable codes to automatically adapt to new computer architectures achieve maximum architecture independence and efficiently use complex libraries.

Integrated Development Environments (IDEs) -- Integrated frameworks supporting the effective integration of development and runtime environments and achieve significant improvements in programmer productivity in the creation of complex application codes.

References

These example research topics represent only a portion of the research challenges for petascale tools. All interested proposers are strongly encouraged to study the following references for additional discussion insight:

Software Development Tools for Petascale Computing Workshop Presentations: http://www.csm.ornl.gov/workshops/Petascale07/presentations.html

Software Development Tools for Petascale Computing Workshop Final Report: http://www.csm.ornl.gov/workshops/Petascale07/sdtpc_workshop_report.pdf

Community Building

An important goal of this notice is to foster active, integrated research community in petascale tools for high end systems. Consequently the following are mandatory requirements for awardees:

• All developed code must be released under the most permissive open source license possible. This is to enable other researchers and vendors to build upon research successes with a minimum of intellectual property issues.

• Each research team should plan to send representatives to annual PI meetings and give presentations on the status and promise of their research. Meeting attendees will include invited participates from other relevant research communities. The objectives of these meetings include fostering a sense of community and serving as a venue for exchange of information with complementary programs including the DARPA HPCS program, NSF programs in CISE and OCI, NNSA ASC program, and the DOE/SC SciDAC program.

Proposals should provide a plan for utilizing leadership class systems at Oak Ridge National Laboratory and Argonne National Laboratory and systems at the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory for the purpose of software testing at scale. Each proposal should contain a section which discusses the characteristics of the test environments necessary for the research and identify the time frames in which specific testbed support will be required. Since relatively limited amounts of testing time will be available on these systems, the individual testing plans will be used to develop an overall test plan for the program.

Program Funding

It is anticipated that up to $3 million annually will be available for multiple awards for this program. Awards are planned to be made in Fiscal Year 2009, and proposals may request project support for up to three years. All awards are contingent on the availability of funds and programmatic needs. Annual budgets for successful projects are expected to range from $250,000 to $700,000 per project although smaller projects of exceptional merit may be considered. Annual budgets may increase in the out-years but should remain within the overall annual maximum guidance.

DOE is under no obligation to pay for any costs associated with the preparation or submission of an application. DOE reserves the right to fund, in whole or part, any, all, or none of the applications submitted in response to this Notice.

Proposals from National Laboratories submitted to the Office of Science (SC) as a result of this program announcement will follow the Department of Energy Field Work Proposal process with additional information requested to allow for scientific/technical merit review. The following guidelines for content and format are intended to facilitate an understanding of the requirements necessary for SC to conduct a merit review of a proposal. Please follow the guidelines carefully, as deviations could be cause for declination of a proposal without merit review.

1. Evaluation Criteria

Proposals will be subjected to formal merit review (peer review) and will be evaluated against the following criteria which are listed in descending order of importance:

1. Scientific and/or technical merit of the project;
2. Appropriateness of the proposed method or approach;
3. Competency of applicant's personnel and adequacy of proposed resources; and
4. Reasonableness and appropriateness of the proposed budget.

2. Summary of Proposal Contents

• Field Work Proposal (FWP) Format (Reference DOE Order 412.1A) (DOE ONLY)
• Proposal Cover Page
• Table of Contents
• Budget (DOE Form 4620.1) and Budget Explanation
• Abstract (one page)
• Narrative (main technical portion of the proposal, including background/introduction, proposed research and methods, timetable of activities, and responsibilities of key project personnel)
• Literature Cited
• Biographical Sketch(es)
• Description of Facilities and Resources
• Other Support of Investigator(s)
• Appendix (optional)

A complete formal FWP in a single Portable Document Format (PDF) document that has 'formatted text and graphics' (also known as "native' PDF) must be submitted using the Office of Science Field Work Proposal Instructions provided in the Notice to Users section on the ePMA home page: http://epma.energy.gov. (This submission process includes sending the FWP via CD, with 2 hard copies, using Federal Express).

Please send the CD and 2 hard copies via Federal Express to:

Petascale Tools
c/o Dr. Frederick Johnson
Computer Science Program, SC-21.1
Office of Advanced Scientific Computing Research
Office of Science
19901 Germantown Road
Germantown, MD 20874-1290
ATTN: Program Announcement LAB 08-19

3. Detailed Contents of the Proposal

Adherence to type size and line spacing requirements is necessary for several reasons. No researcher should have the advantage, or by using small type, of providing more text in their proposals. Small type may also make it difficult for reviewers to read the proposal. Proposals must have 1-inch margins at the top, bottom, and on each side. Type sizes must be 11 point. Line spacing is at the discretion of the researcher but there must be no more than 6 lines per vertical inch of text. Pages should be standard 8 1/2" x 11" (or metric A4, i.e., 210 mm x 297 mm).

3.1 Field Work Proposal Format (Reference DOE Order 412.1A)

The Field Work Proposal (FWP) is to be prepared and submitted consistent with policies of the investigator's laboratory and the local DOE Operations Office. Additional information is also requested to allow for scientific/technical merit review. Laboratories may submit proposals directly to the SC Program office listed above. A copy should also be provided to the appropriate DOE operations office.

3.2 Proposal Cover Page

The following proposal cover page information may be placed on plain paper. No form is required.

Title of proposed project
SC Program announcement title
Name of laboratory
Name of principal investigator (PI)
Position title of PI
Mailing address of PI
Telephone of PI
Fax number of PI
Electronic mail address of PI
Name of official signing for laboratory*
Title of official
Fax number of official
Telephone of official
Electronic mail address of official
Requested funding for each year; total request
Use of human subjects in proposed project:
If activities involving human subjects are not planned at any time during the proposed project period, state "No"; otherwise state "Yes", provide the IRB Approval date and Assurance of Compliance Number and include all necessary information with the proposal should human subjects be involved.
Use of vertebrate animals in proposed project:
If activities involving vertebrate animals are not planned at any time during this project, state "No"; otherwise state "Yes" and provide the IACUC Approval date and Animal Welfare Assurance number from NIH and include all necessary information with the proposal.
Signature of PI, date of signature
Signature of official, date of signature*

3.3 Table of Contents

Provide the initial page number for each of the sections of the proposal. Number pages consecutively at the bottom of each page throughout the proposal. Start each major section at the top of a new page. Do not use unnumbered pages and do not use suffices, such as 5a, 5b.

3.4 Budget and Budget Explanation

A detailed budget is required for each fiscal year. It is preferred that DOE's budget page, Form 4620.1 be used for providing budget information*. Modifications of categories are permissible to comply with institutional practices, for example with regard to overhead costs.

A written justification of each budget item is to follow the budget pages. For personnel this should take the form of a one-sentence statement of the role of the person in the project. Provide a detailed justification of the need for each item of permanent equipment. Explain each of the other direct costs in sufficient detail for reviewers to be able to judge the appropriateness of the amount requested.

Further instructions regarding the budget are given in section 4 of this guide.

* Form 4620.1 is available at web site: http://www.science.doe.gov/grants/budgetform.pdf

3.5 Abstract

Provide an abstract of less than 400 words. Give the project objectives (in broad scientific terms), the approach to be used, and what the research is intended to accomplish. State the hypotheses to be tested (if any). At the top of the abstract give the project title, names of all the investigators and their institutions, and contact information for the principal investigator, including e-mail address.

3.6 Narrative (main technical portion of the proposal, including background/introduction, proposed research and methods, timetable of activities, and responsibilities of key project personnel).

The narrative comprises the research plan for the project and is limited to 13 pages (maximum). It should contain enough background material in the Introduction, including review of the relevant literature, to demonstrate sufficient knowledge of the state of the science. The major part of the narrative should be devoted to a description and justification of the proposed project, including details of the methods to be used. It should also include a timeline for the major activities of the proposed project, and should indicate which project personnel will be responsible for which activities.

If any portion of the project is to be done in collaboration with another institution (or institutions), provide information on the institution(s) and what part of the project it will carry out. Further information on any such arrangements is to be given in the sections "Budget and Budget Explanation," "Biographical Sketches," and "Description of Facilities and Resources."

3.7 Biographical Sketches

This information is required for senior personnel at the institution submitting the proposal and at all subcontracting institutions (if any). The biographical sketch is limited to a maximum of two pages for each investigator.

To assist in the identification of potential conflicts of interest or bias in the selection of reviewers, the following information must be provided in each biographical sketch.

Collaborators and Co-editors: A list of all persons in alphabetical order (including their current organizational affiliations) who are currently, or who have been, collaborators or co- authors with the investigator on a research project, book or book article, report, abstract, or paper during the 48 months preceding the submission of the proposal. Also include those individuals who are currently or have been co-editors of a special issue of a journal, compendium, or conference proceedings during the 24 months preceding the submission of the proposal. If there are no collaborators or co- editors to report, this should be so indicated.

Graduate and Postdoctoral Advisors and Advisees: A list of the names of the individual's own graduate advisor(s) and principal postdoctoral sponsor(s), and their current organizational affiliations. A list of the names of the individual's graduate students and postdoctoral associates during the past five years, and their current organizational affiliations.

Facilities to be used for the conduct of the proposed research should be briefly described. Indicate the pertinent capabilities of the institution, including support facilities (such as machine shops), that will be used during the project. List the most important equipment items already available for the project and their pertinent capabilities. Include this information for each subcontracting institution (if any).

3.9 Statement of all Current and Pending Support

Other support is defined as all financial resources, whether Federal, non-Federal, commercial, or institutional, available in direct support of an individual's research endeavors. Information on active and pending other support is required for all senior personnel, including investigators at collaborating institutions to be funded by a subcontract. For each item of other support, give the organization or agency, inclusive dates of the project or proposed project, annual funding, and level of effort (months per year or percentage of the year) devoted to the project.

3.10 Appendix (optional)

Information not easily accessible to a reviewer may be included in an appendix. Reviewers are not required to consider information in an appendix, and reviewers may not have time to read extensive appendix materials with the same care they would use with the proposal proper. The appendix may contain the following items: up to five publications, manuscripts accepted for publication, abstracts, patents, or other printed materials directly relevant to this project, but not generally available to the scientific community; and letters from investigators at other institutions stating their agreement to participate in the project (do not include letters of endorsement of the project).

4. Detailed Instructions for the Budget
(DOE Form 4620.1 "Budget Page" may be used).

4.1 Salaries and Wages

List the names of the principal investigator and other key personnel and the estimated number of person-months for which DOE funding is requested. Proposers should list the number of postdoctoral associates and other professional positions included in the proposal and indicate the number of full-time-equivalent (FTE) person-months and rate of pay (hourly, monthly or annually). For graduate and undergraduate students and all other personnel categories such as secretarial, clerical, technical, etc., show the total number of people needed in each job title and total salaries needed. Salaries requested must be consistent with the institution's regular practices. The budget explanation should define concisely the role of each position in the overall project.

4.2 Equipment

DOE defines equipment as "an item of tangible personal property that has a useful life of more than two years and an acquisition cost of $25,000 or more." Special purpose equipment means equipment which is used only for research, scientific or other technical activities. Items of needed equipment should be individually listed by description and estimated cost, including tax, and adequately justified. Allowable items ordinarily will be limited to scientific equipment that is not already available for the conduct of the work. General purpose office equipment normally will not be considered eligible for support.

4.3 Domestic Travel

The type and extent of travel and its relation to the research should be specified. Funds may be requested for attendance at meetings and conferences, other travel associated with the work and subsistence. In order to qualify for support, attendance at meetings or conferences must enhance the investigator's capability to perform the research, plan extensions of it, or disseminate its results. Consultant's travel costs also may be requested.

4.4 Foreign Travel

Foreign travel is any travel outside Canada and the United States and its territories and possessions. Foreign travel may be approved only if it is directly related to project objectives.

4.5 Other Direct Costs

The budget should itemize other anticipated direct costs not included under the headings above, including materials and supplies, publication costs, computer services, and consultant services (which are discussed below). Other examples are: aircraft rental, space rental at research establishments away from the institution, minor building alterations, service charges, and fabrication of equipment or systems not available off- the-shelf. Reference books and periodicals may be charged to the project only if they are specifically related to the research.

a. Materials and Supplies

The budget should indicate in general terms the type of required expendable materials and supplies with their estimated costs. The breakdown should be more detailed when the cost is substantial.

b. Publication Costs/Page Charges

The budget may request funds for the costs of preparing and publishing the results of research, including costs of reports, reprints page charges, or other journal costs (except costs for prior or early publication), and necessary illustrations.

c. Consultant Services

Anticipated consultant services should be justified and information furnished on each individual's expertise, primary organizational affiliation, daily compensation rate and number of days expected service. Consultant's travel costs should be listed separately under travel in the budget.

d. Computer Services

The cost of computer services, including computer-based retrieval of scientific and technical information, may be requested. A justification based on the established computer service rates should be included.

e. Subcontracts

Subcontracts should be listed so that they can be properly evaluated. There should be an anticipated cost and an explanation of that cost for each subcontract. The total amount of each subcontract should also appear as a budget item.

4.6 Indirect Costs

Explain the basis for each overhead and indirect cost. Include the current rates.