U.S.
DEPARTMENT OF
ENERGY

For more information about the Office of Science, go to Office of Science

Program Announcement
To DOE National Laboratories
LAB 07-09

Scientific Discovery Through
Advanced Computing: Accelerator
Science and Simulation

SUMMARY: The Offices of Advanced Scientific Computing Research (ASCR), High Energy Physics (HEP), Nuclear Physics (NP), and Basic Energy Sciences (BES), U.S. Department of Energy (DOE), hereby announce their interest in receiving proposals for basic Research and Development (R&D) projects relevant to the development of advanced simulations of particle accelerators and associated technologies. This research should provide the capability for comprehensive 3-dimensional, end-to-end modeling of existing and proposed particle accelerators, and advanced acceleration concepts, particularly aimed at HEP and NP proposals, to optimize operations of current facilities and the design of future facilities. Subject to appropriations, funds are anticipated to be available within the SciDAC programs of ASCR, HEP, and NP to support this research. DATES: Full proposals submitted in response to this Announcement must be submitted to the DOE Electronic Proposal Management Application (ePMA) system (https://epma.doe.gov) no later than 8:00 p.m., Eastern Time, January 17, 2007, to be accepted for merit review and to permit timely consideration for award in Fiscal Year 2007. It is important that the entire peer reviewable proposal be submitted to the ePMA system as a single PDF file attachment.

Please see the "Addresses" section below for further instructions on the methods of submission for the full proposal.

ADDRESSES: A complete formal FWP in a single Portable Document Format (PDF) file must be submitted through the DOE ePMA system (https://epma.doe.gov) as an attachment. To identify that the FWP is responding to this program announcement, please fill in the following fields in the "ePMA Create Proposal Admin Information" screen as shown:

    Proposal Short Name:
    Fiscal Year:
    Proposal Reason:
    Program Announcement Number:
    LAB 07-09 *
    Program announcement Title: Scientific Discovery Through Advanced Computing: Accelerator Science and Simulation *
    Proposal Purpose:
    Estimated Proposal Begin Date:
    HQ Program Manager Organization:

    * Please use the wording shown when filling in these fields to identify that the FWP is responding to this Program Announcement.

In order to expedite the review process, please submit via email a single PDF file of the entire LAB proposal and FWP. The email should be addressed to Ms. Donna Lang at: Donna.Lang@science.doe.gov. Please use "Program Announcement LAB 07-09" as the subject of the email.

DOE National Laboratories should submit using ePMA as instructed above. Researchers from other Federal agencies and Non-DOE Federally Funded Research and Development Centers (FFRDCs) should follow the format at http://www.science.doe.gov/grants/fed_prop.html and submit via email as stated above.

FOR FURTHER INFORMATION CONTACT:

Dr. Glen Crawford, Office of High Energy Physics
PHONE: (301) 903-9458
FAX:      (301) 903-2597
E-MAIL: Glen.Crawford@science.doe.gov

Dr. Craig Tull, Office of High Energy Physics
PHONE: (301) 903-0468
FAX:      (301) 903-2597
E-MAIL: Craig.Tull@science.doe.gov

Dr. Sidney Coon, Office of Nuclear Physics
PHONE: (301) 903-7878
FAX:      (301) 903-3833
EMAIL: Sidney.A.Coon@science.doe.gov

Dr. Roger Klaffky, Office of Basic Energy Sciences
PHONE: (301) 903-1873
FAX:      (301) 903-1690
EMAIL: Roger.Klaffky@science.doe.gov

Dr. Walter Polansky, Computational Science Research and Partnerships (SciDAC) Division
PHONE: (301) 903-0192
FAX:      (301) 903-0365
EMAIL: walt.polansky@science.doe.gov

Communications related to the formal proposal should use "Program Announcement LAB 07-09" in the subject line.

SUPPLEMENTARY INFORMATION: Background: Scientific Discovery Through Advanced Computing Program (SciDAC)

The Scientific Discovery through Advanced Computing (SciDAC) program was initiated in 2001 as a partnership involving all SC program offices - Advanced Scientific Computing Research, Basic Energy Sciences, Biological and Environmental Research, Fusion Energy Sciences, High-Energy Physics and Nuclear Physics - to fully realize the potential of emerging high performance computers at that time for advancing scientific discovery. Researchers have achieved key scientific insights in a number of areas of National importance, yet many challenges of multi-scale, multi-disciplinary problems now facing science programs in DOE require advanced modeling and simulation capabilities on petascale computers. SciDAC projects should:

  • Address obtaining significant insight into, or actually solve, a challenging problem of National scientific or engineering significance clearly related to DOE missions through computational science,
  • Create open source scientific simulation codes that: achieve high single node performance; scale to thousands of nodes and tens-of-thousands of processors; and can be readily ported to other computer architectures,
  • Develop applied mathematics and computer science methodology focused on computational science at the petascale and work with application scientists to apply innovations,
  • Integrate computational science with discipline-driven applications through teaming and partnerships with computer scientists and applied mathematicians,
  • Engage experimental and observational data-intensive science, and/or
  • Empower new scientific communities to achieve scientific discovery through computational science.
Proposals

Accelerator Science and Simulation: A comprehensive, coherent petascale simulation capability for the U.S. accelerator community is critical for the near and long-term priorities of DOE's Office of Science. We desire proposals that can deliver a wide range of simulation capabilities in accelerator physics, and in particular those that broadly address the needs and priorities of the High Energy Physics and Nuclear Physics communities, and can integrate electrodynamic modeling of accelerator structures with end-to-end beam dynamics and propagation. Proposals that are limited to only particular phenomena or narrow aspects of accelerator physics will receive lower priority for funding.

High Energy Physics priorities in accelerator simulation are driven by optimization needs of existing HEP accelerators, such as the B-Factory and Tevatron; design of possible future accelerators, such as International Linear Collider (ILC) and other next-generation facilities including the neutrino factory / muon collider concept; and maintaining a vital DOE accelerator R&D program. Nuclear Physics priorities are the development of a design for a US rare isotope beam capabilities and maintaining an accelerator R&D development plan which includes the needs of the Continuous Electron Beam Accelerator Facility (CEBAF) Upgrade, electron cooling required for the luminosity upgrade of the Relativistic Heavy Ion Collider and, in the future, addresses the exploration of an electron-ion collider that would allow the gluon saturation of nuclear matter to be seen.

Basic Energy Sciences priorities include the optimization and upgrade of the accelerator- based BES light sources and neutron scattering facilities. In the design and commissioning of new Free Electron Lasers (FELS) such as the Linac Coherent Light Source (LCLS) simulations of the creation and transport of high brightness electron beams are another BES priority.

Topic areas for advanced modeling include, but are not limited to: high-accuracy computation of modes for superconducting RF cavities; realistic simulation of wakefield effects; parallelization of Radio Frequency Quadrupole simulations; self-consistent 3D calculations of Coherent Synchrotron Radiation, forces and their effects on the beam; electron cooling of beams; heavy ion transport: optimization of Particle-In-Cell codes; and adaptive mesh techniques for intense beams. Accelerator simulation codes which run on a variety of platforms; scale to petaflops and many thousands of processors; which are robust, documented, and can be easily used by the community of accelerator researchers; and are well integrated with visualization capacities will have the greatest impact on the field. Proposals should address how they will meet these criteria.

In addition, proposals must describe the computational approach for interacting with the SciDAC Institutes and Centers for Enabling Technologies. At a minimum, this description should include:

  • Programming languages, libraries, and other software used.
  • Description of the underlying mathematical formulation (e.g., ODE, PDE).
  • Algorithms and numerical techniques employed (e.g., finite element, iterative solver).
  • Parallel programming system used (e.g., MPI, OpenMP, "embarrassingly" parallel).
  • Or data-intensive applications, describe the data storage and transfer requirements.
Partnerships among multiple institutions, that may include universities, laboratories, and/or private institutions, are anticipated for the scientific selection teams of the submissions.

In view of the wide applicability of the simulation codes called for in this request, and a potentially large group of users with different needs and requirements, we expect that successful proposals will incorporate a clear management structure, as well as deliver the tools needed by the scientific applications in a timely fashion. The scientific teams should propose a program of work that achieves their goals consistent with this management structure, and aligned with the priorities of the High Energy Physics and Nuclear Physics programs as outlined above.

Proposals that do not include a formal collaboration with one, or more, SciDAC Centers for Enabling Technology will not be considered for award. The Centers for Enabling Technology are authorized to enter into such collaborations within their existing scope and budget. No funds are to be requested under this Notice for sub-contracts, or task orders, to a Center for Enabling Technology for participation in a proposed collaboration.

Researchers must include a separate budget and statement for each proposed funding source. Successful researchers are not required to request funding from all four participating Office of Science programs; however, proposed funding sources should align with the specific nature of the work being performed, as documented in the proposal, and reflect the collaborative nature of the SciDAC program.

Collaboration

Proposers are encouraged to collaborate with researchers in other institutions, such as: universities, industry, non-profit organizations, federal laboratories and Federally Funded Research and Development Centers (FFRDCs), including the DOE National Laboratories, where appropriate, and to include cost sharing and/or consortia wherever feasible. All collaborators should be listed with the abstract or summary. Additional information on collaboration is available in the Application Guide for the Office of Science Financial Assistance Program that is available via the Internet at the web address: http://www.science.doe.gov/grants/Colab.html.

Program Funding

It is anticipated that up to $3,500,000 will be available for multiple awards to be made in Fiscal Year 2007, in the areas described above, contingent on the availability of appropriated funds. It is anticipated that project selection will be completed by April 30, 2007.

Proposals should be for one to three years, with a continuation of up to two additional years possible for those tasks requiring a multi-year effort. Given the scope of the R&D effort, we anticipate most proposals will be multi-year. For continuation of multi-year effort, out-year support is contingent on the availability of funds, progress of the research and programmatic needs. Formal Proposals The research project description must be 15 pages or less, exclusive of attachments and must contain an abstract or summary of the proposed research. All collaborators should be listed with the abstract or summary. Attachments include curriculum vitae, a listing of all current and pending federal support and letters of intent when collaborations are part of the proposed research. Curriculum vitae should be limited to no more than two pages per individual.

The instructions and format described below should be followed. You must reference Program Announcement LAB 07-09 on all submissions and inquiries about this program.

OFFICE OF SCIENCE
GUIDE FOR PREPARATION OF SCIENTIFIC/TECHNICAL PROPOSALS
TO BE SUBMITTED BY NATIONAL LABORATORIES

Proposals from National Laboratories submitted to the Office of Science (SC) as a result of this program announcement will follow the Department of Energy Field Work Proposal process with additional information requested to allow for scientific/technical merit review. The following guidelines for content and format are intended to facilitate an understanding of the requirements necessary for SC to conduct a merit review of a proposal. Please follow the guidelines carefully, as deviations could be cause for declination of a proposal without merit review.

1. Evaluation Criteria

Proposals will be subjected to formal merit review (peer review) and will be evaluated against the following criteria which are listed in descending order of importance:

    1. Scientific and/or technical merit of the project;
    2. Appropriateness of the proposed method or approach;
    3. Competency of the personnel and adequacy of the proposed resources; and
    4. Reasonableness and appropriateness of the proposed budget.
The evaluation will include program policy factors such as the relevance of the proposed research to the terms of the announcement, the Department's programmatic needs, and quality of previous performance. External peer reviewers are selected with regard to both their scientific expertise and the absence of conflict-of-interest issues. Non-federal reviewers may be used, and submission of a proposal constitutes agreement that this is acceptable to the investigator(s) and the submitting institution. Proposals found to be scientifically meritorious and programmatically relevant will be selected in consultation with DOE selecting officials depending upon availability of funds in the DOE budget. The selected projects will be required to acknowledge support by DOE in all public communications of the research results.

2. Summary of Proposal Contents

  • Field Work Proposal (FWP) Format (Reference DOE O 412.1A) (DOE ONLY)
  • Proposal Cover Page
  • Table of Contents
  • Budget (DOE Form 4620.1) and Budget Explanation
  • Abstract (one page)
  • Narrative (main technical portion of the proposal, including background/introduction, proposed research and methods, timetable of activities, and responsibilities of key project personnel)
  • Literature Cited
  • Biographical Sketch(es)
  • Description of Facilities and Resources
  • Other Support of Investigator(s)
  • Appendix (optional)
2.1 Number of Copies to Submit

A complete formal FWP in a single Portable Document Format (PDF) file must be submitted through the DOE ePMA system (https://epma.doe.gov) as an attachment. To identify that the FWP is responding to this program announcement, please fill in the following fields in the "ePMA Create Proposal Admin Information" screen as shown:

    Proposal Short Name:
    Fiscal Year:
    Proposal Reason:
    Program Announcement Number:
    LAB 07-09 *
    Program announcement Title: Scientific Discovery Through Advanced Computing: Accelerator Science and Simulation *
    Proposal Purpose:
    Estimated Proposal Begin Date:
    HQ Program Manager Organization:

    * Please use the wording shown when filling in these fields to identify that the FWP is responding to this Program Announcement.

In order to expedite the review process, please submit via email a single PDF file of the entire LAB proposal and FWP. The email should be addressed to Ms. Donna Lang at: Donna.Lang@science.doe.gov. Please use "Program Announcement LAB 07-09" as the subject of the email.

3. Detailed Contents of the Proposal

Adherence to type size and line spacing requirements is necessary for several reasons. No researcher should have the advantage, or by using small type, of providing more text in their proposals. Small type may also make it difficult for reviewers to read the proposal. Proposals must have 1-inch margins at the top, bottom, and on each side. Type sizes must be at least 11 point. Line spacing is at the discretion of the researcher but there must be no more than 6 lines per vertical inch of text. Pages should be standard 8 1/2" x 11" (or metric A4, i.e., 210 mm x 297 mm).

3.1 Field Work Proposal Format (Reference DOE O 412.1A) (DOE ONLY)

The Field Work Proposal (FWP) is to be prepared and submitted consistent with policies of the investigator's laboratory and the local DOE Operations Office. Additional information is also requested to allow for scientific/technical merit review.

Laboratories may submit proposals directly to the SC Program office listed above. A copy should also be provided to the appropriate DOE operations office.

3.2 Proposal Cover Page

The following proposal cover page information may be placed on plain paper. No form is required. The following proposal cover page information may be placed on plain paper. No form is required.

    Title of proposed project
    SC Program announcement title
    Name of laboratory
    Name of principal investigator (PI)
    Position title of PI
    Mailing address of PI
    Telephone of PI
    Fax number of PI
    Electronic mail address of PI
    Name of official signing for laboratory*
    Title of official
    Fax number of official
    Telephone of official
    Electronic mail address of official
    Requested funding for each year; total request
    Use of human subjects in proposed project:
      If activities involving human subjects are not planned at any time during the proposed project period, state "No"; otherwise state "Yes", provide the IRB Approval date and Assurance of Compliance Number and include all necessary information with the proposal should human subjects be involved.
    Use of vertebrate animals in proposed project:
      If activities involving vertebrate animals are not planned at any time during this project, state "No"; otherwise state "Yes" and provide the IACUC Approval date and Animal Welfare Assurance number from NIH and include all necessary information with the proposal.
    Signature of PI, date of signature
    Signature of official, date of signature*

    *The signature certifies that personnel and facilities are available as stated in the proposal, if the project is funded.

3.3 Table of Contents

Provide the initial page number for each of the sections of the proposal. Number pages consecutively at the bottom of each page throughout the proposal. Start each major section at the top of a new page. Do not use unnumbered pages and do not use suffices, such as 5a, 5b.

Provide the initial page number for each of the sections of the proposal. Number pages consecutively at the bottom of each page throughout the proposal. Start each major section at the top of a new page. Do not use unnumbered pages and do not use suffices, such as 5a, 5b.

3.4 Budget and Budget Explanation

A detailed budget is required for the entire project period and for each fiscal year. It is preferred that DOE's budget page, Form 4620.1 be used for providing budget information*. Modifications of categories are permissible to comply with institutional practices, for example with regard to overhead costs.

A written justification of each budget item is to follow the budget pages. For personnel this should take the form of a one-sentence statement of the role of the person in the project. Provide a detailed justification of the need for each item of permanent equipment. Explain each of the other direct costs in sufficient detail for reviewers to be able to judge the appropriateness of the amount requested.

Further instructions regarding the budget are given in section 4 of this guide.

* Form 4620.1 is available at web site: http://www.science.doe.gov/grants/budgetform.pdf

3.5 Abstract

Provide an abstract of less than 400 words. Give the project objectives (in broad scientific terms), the approach to be used, and what the research is intended to accomplish. State the hypotheses to be tested (if any). At the top of the abstract give the project title, names of all the investigators and their institutions, and contact information for the principal investigator, including e-mail address.

3.6 Narrative (main technical portion of the proposal, including background/introduction, proposed research and methods, timetable of activities, and responsibilities of key project personnel).

The narrative comprises the research plan for the project and is limited to
5 pages (maximum). It should contain enough background material in the Introduction, including review of the relevant literature, to demonstrate sufficient knowledge of the state of the science. The major part of the narrative should be devoted to a description and justification of the proposed project, including details of the methods to be used. It should also include a timeline for the major activities of the proposed project, and should indicate which project personnel will be responsible for which activities.

If any portion of the project is to be done in collaboration with another institution (or institutions), provide information on the institution(s) and what part of the project it will carry out. Further information on any such arrangements is to be given in the sections "Budget and Budget Explanation", "Biographical Sketches", and "Description of Facilities and Resources".

3.7 Literature Cited

Give full bibliographic entries for each publication cited in the narrative.

3.8 Biographical Sketches

This information is required for senior personnel at the institution submitting the proposal and at all subcontracting institutions (if any). The biographical sketch is limited to a maximum of two pages for each investigator.

To assist in the identification of potential conflicts of interest or bias in the selection of reviewers, the following information must be provided in each biographical sketch.

    Collaborators and Co-editors: A list of all persons in alphabetical order (including their current organizational affiliations) who are currently, or who have been, collaborators or co- authors with the investigator on a research project, book or book article, report, abstract, or paper during the 48 months preceding the submission of the proposal. Also, include those individuals who are currently or have been co-editors of a special issue of a journal, compendium, or conference proceedings during the 24 months preceding the submission of the proposal. If there are no collaborators or co-editors to report, this should be so indicated.

    Graduate and Postdoctoral Advisors and Advisees: A list of the names of the individual's own graduate advisor(s) and principal postdoctoral sponsor(s), and their current organizational affiliations. A list of the names of the individual's graduate students and postdoctoral associates during the past five years, and their current organizational affiliations.

3.9 Description of Facilities and Resources

Facilities to be used for the conduct of the proposed research should be briefly described. Indicate the pertinent capabilities of the institution, including support facilities (such as machine shops), that will be used during the project. List the most important equipment items already available for the project and their pertinent capabilities. Include this information for each subcontracting institution (if any).

3.10 Other Support of Investigators

Other support is defined as all financial resources, whether Federal, non-Federal, commercial, or institutional, available in direct support of an individual's research endeavors. Information on active and pending other support is required for all senior personnel, including investigators at collaborating institutions to be funded by a subcontract. For each item of other support, give the organization or agency, inclusive dates of the project or proposed project, annual funding, and level of effort (months per year or percentage of the year) devoted to the project.

3.11 Appendix

Information not easily accessible to a reviewer may be included in an appendix, but do not use the appendix to circumvent the page limitations of the proposal. Reviewers are not required to consider information in an appendix, and reviewers may not have time to read extensive appendix materials with the same care they would use with the proposal proper.

The appendix may contain the following items: up to five publications, manuscripts accepted for publication, abstracts, patents, or other printed materials directly relevant to this project, but not generally available to the scientific community; and letters from investigators at other institutions stating their agreement to participate in the project (do not include letters of endorsement of the project).

4. Detailed Instructions for the Budget
(DOE Form 4620.1 "Budget Page" may be used)

4.1 Salaries and Wages

List the names of the principal investigator and other key personnel and the estimated number of person-months for which DOE funding is requested. Proposers should list the number of postdoctoral associates and other professional positions included in the proposal and indicate the number of full-time-equivalent (FTE) person-months and rate of pay (hourly, monthly or annually). For graduate and undergraduate students and all other personnel categories such as secretarial, clerical, technical, etc., show the total number of people needed in each job title and total salaries needed. Salaries requested must be consistent with the institution's regular practices. The budget explanation should define concisely the role of each position in the overall project.

4.2 Equipment

DOE defines equipment as "an item of tangible personal property that has a useful life of more than two years and an acquisition cost of $25,000 or more." Special purpose equipment means equipment which is used only for research, scientific or other technical activities. Items of needed equipment should be individually listed by description and estimated cost, including tax, and adequately justified. Allowable items ordinarily will be limited to scientific equipment that is not already available for the conduct of the work. General purpose office equipment normally will not be considered eligible for support.

4.3 Domestic Travel

The type and extent of travel and its relation to the research should be specified. Funds may be requested for attendance at meetings and conferences, other travel associated with the work and subsistence. In order to qualify for support, attendance at meetings or conferences must enhance the investigator's capability to perform the research, plan extensions of it, or disseminate its results. Consultant's travel costs also may be requested.

4.4 Foreign Travel

Foreign travel is any travel outside Canada and the United States and its territories and possessions. Foreign travel may be approved only if it is directly related to project objectives.

4.5 Other Direct Costs

The budget should itemize other anticipated direct costs not included under the headings above, including materials and supplies, publication costs, computer services, and consultant services (which are discussed below). Other examples are: aircraft rental, space rental at research establishments away from the institution, minor building alterations, service charges, and fabrication of equipment or systems not available off- the-shelf. Reference books and periodicals may be charged to the project only if they are specifically related to the research.

a. Materials and Supplies

The budget should indicate in general terms the type of required expendable materials and supplies with their estimated costs. The breakdown should be more detailed when the cost is substantial.

b. Publication Costs/Page Charges

The budget may request funds for the costs of preparing and publishing the results of research, including costs of reports, reprints page charges, or other journal costs (except costs for prior or early publication), and necessary illustrations.

c. Consultant Services

Anticipated consultant services should be justified and information furnished on each individual's expertise, primary organizational affiliation, daily compensation rate and number of days expected service. Consultant's travel costs should be listed separately under travel in the budget.

d. Computer Services

The cost of computer services, including computer-based retrieval of scientific and technical information, may be requested. A justification based on the established computer service rates should be included.

e. Subcontracts

Subcontracts should be listed so that they can be properly evaluated. There should be an anticipated cost and an explanation of that cost for each subcontract. The total amount of each subcontract should also appear as a budget item.

4.6 Indirect Costs

Explain the basis for each overhead and indirect cost. Include the current rates.