A QUALITY ASSESSMENT CENTER FOR GENOMIC DNA SEQUENCE

Release Date:  December 15, 1999

RFA:  HG-00-001 

National Human Genome Research Institute

Letter of Intent Receipt Date: January 12, 2000
Application Receipt Date:  March 14, 2000

PURPOSE

The objective of this RFA is to establish an independent DNA sequence quality 
assessment center (QAC) to evaluate the quality of the DNA sequence being 
produced with NHGRI support.  On a regular basis, the QAC will conduct 
assessments of the quality of the genomic sequence data being produced by each 
of the NHGRI-funded centers involved in the production of human and/or mouse 
genomic DNA sequence.  Regular assessments of genomic sequence data are 
essential to assure the scientific community and the public that the quality 
of the sequence being deposited in the public databases meets the 
international standards and provides a reliable basis for further biomedical 
research. Assessments will also provide fundamental data about the quality 
characteristics of the sequence data, which can be used by the scientific 
community as an aid to interpreting the sequence data and using it to design 
further experiments and improved analyses.  As a secondary goal, the QAC will 
be a resource to the sequencing centers, and to the National Human Genome 
Research Institute (NHGRI) for technical advice about sequence quality 
assessment.  The NHGRI is also open to the possibility that the QAC will 
undertake, as an additional, but secondary objective, a research and 
development effort to improve methods of quality assessment for genomic 
sequence.

HEALTHY PEOPLE 2000

The Public Health Service (PHS) is committed to achieving the health promotion 
and disease prevention objectives of "Healthy People 2000," a PHS-led national 
activity for setting priority areas.  This RFA, A Quality Assessment Center 
for Genomic DNA Sequence, is related to several priority areas including 
cancer, heart disease and stroke, diabetes and chronic disability conditions, 
and maternal and infant health. Potential applicants may obtain a copy of 
"Healthy People 2000" at http://odphp.osophs.dhhs.gov/pubs/hp2000.

ELIGIBILITY REQUIREMENTS

Applications may be submitted by domestic, for-profit and non-profit 
organizations, public and private, such as universities, colleges, hospitals, 
laboratories, units of State and local governments, and eligible agencies of 
the Federal government. Applications from foreign institutions will not be 
accepted; however subcontracts to foreign institutions will be considered. 
Racial/ethnic minority individuals, women, and persons with disabilities are 
encouraged to apply as Principal Investigators. A principal investigator 
submitting an application for the QAC must be able to demonstrate independence 
from any NHGRI-funded large-scale sequencing center whose data are to be 
evaluated by the QAC. 

MECHANISM OF SUPPORT

The administrative and funding mechanism to be used to support this program 
will be the Cooperative Agreement (U01), an "assistance" mechanism, which is 
distinguished from a regular research grant in that substantial scientific 
and/or programmatic involvement by NHGRI staff with the awardee is 
anticipated.  The cooperative agreement is used when participation by NIH 
staff is warranted to support and/or stimulate the recipient's activity by 
involvement in and otherwise working jointly with the award recipient in a 
partner role; NIH staff will not assume direction, primary responsibility, or 
a dominant role in the activity.  Details of the responsibilities, 
relationships, and governance of the studies funded under cooperative 
agreement(s) are discussed later in this document under the section Terms and 
Conditions of Award. 

The initial award for this RFA will be for three years; it is anticipated that 
the need for a QAC will persist as long as NHGRI funds large-scale sequence 
production. 

FUNDS AVAILABLE

The estimated funds available for the first year of support for awards under 
this RFA will be $2.8 million per year (total costs). One award will be made. 

The usual NIH policies governing grants administration and management will 
apply.  This level of support is dependent on the receipt of a sufficient 
number of applications of high scientific merit.  Beyond the first year, the 
funding level of the QAC will be based on an annual evaluation.  The primary 
criterion for evaluation will be whether the QAC is adequately performing 
regular quality assessments of the NHGRI-funded sequence production centers 
and fulfilling the long- and short-term needs of the human and mouse genomic 
sequencing programs, as determined by the scientific consultants to the large-
scale sequencing program, NHGRI staff and the National Advisory Council for 
Human Genome Research (NACHGR).  The funding level for the QAC will also be 
dependent upon the availability of funds as well as the number of sequence 
production centers.

RESEARCH OBJECTIVES 

Background

The National Human Genome Research Institute (NHGRI) is currently engaged, 
along with several other federal, private, and international organizations, in 
a fifteen-year research program called the Human Genome Project (HGP).  The 
HGP started in 1990 and significant progress toward completing its goals has 
been achieved. The genetic and physical mapping goals for both the human and 
the mouse have been met. There has also been good progress toward meeting the 
sequencing goals. The DNA sequence of both the E. coli and S. cerevisiae 
genomes has been determined (as have those of several other microorganisms), 
the sequence of the C. elegans genome was completed in 1998, and it is 
anticipated that the complete DNA sequence of the D. melanogaster genome will 
be finished by 2001. 

The HGP's primary goal is now the completion of an index human DNA sequence 
(see http://www.nhgri.nih.gov/98plan).  To effect its part of the sequencing 
program, NHGRI initiated a three-year pilot sequencing program in 1996.  This 
was followed in the first half of 1999 by scale up to a full production mode, 
and NHGRI now supports seven cooperative agreements for sequencing the human 
genome.  As of the end of September 1999, the world-wide public genome 
sequencing effort had deposited over 460 Mb of finished human sequence in the 
public databases, plus more than 400 Mb of working draft quality sequence. 

During the past five years, advances and refinements in DNA sequencing have 
led to a rapid acceleration in the level of sequencing capacity. First, 
technological advances (e.g., capillary sequencing machines) have progressed 
even more rapidly than anticipated.  Second, the degree of cooperation among 
the public sequencing laboratories has increased significantly.  Accordingly, 
the schedule for attaining the Human Genome Project's goal of completing an 
index human DNA sequence has been advanced several times.  Completion is now 
projected by 2003.  Furthermore, a working draft version is anticipated by 
Spring 2000, and until then, most of the capacity for human sequencing will be 
devoted to working draft production.  Although primarily an intermediate in 
the generation of a high quality finished human DNA sequence, the working 
draft will itself be useful for many types of biological analyses.  

The primary objective of the sequencing phase of the Human Genome Project is 
to generate a very high quality index sequence of human DNA that will be 
usable as the basis for a wide variety of biological studies for the 
foreseeable future.    After extensive discussion with the scientific 
community, including both sequencers and users of sequence data, the HGP has 
developed a set of standards for the quality of the finished human genomic 
sequence data.  The full text of the sequence quality policy that has been 
adopted by NHGRI in light of these community determinations is available at 
http://www.nhgri.nih.gov/Grant_info/Funding/Statements/RFA/quality_standard.html.
In summary, 

o      the sequence must be at least 99.99% accurate (i.e, fewer than one 
error per 10,000 bases); and 
 o     the sequence should be contiguous (i.e. with no gaps, other than those 
few areas that cannot be cloned in current vectors).

These standards are very stringent, and it has been recognized that they may 
not be achievable in some regions of the genome.  In those cases, any 
deviations from the standard must be documented, including the location and 
size of gaps, the orientation of flanking regions, and a description of the 
efforts made to close the gaps. These standards may be applied to genomic 
sequence from other organisms as NHGRI undertakes to sequence them. 

It is critical for the scientific community to be able to rely on, and be 
informed about, the quality of the genomic sequence. Objective data must be 
available to assure the scientific and general public that the sequence data 
meet the standards. Quantitative information about sequence quality is also 
required for designing further experiments. For example, studies on genetic 
variation, analysis of disease-causing mutations, protein structure 
prediction, comparative sequence analysis, sequence annotation, and sequence 
assembly are all increasingly utilizing information about genomic sequence 
quality. 

Before the final adoption of the sequence quality standards, the NHGRI carried 
out a set of experiments designed to test whether the quality of DNA sequence 
data could be assessed and whether the standards could be achieved.  The 
results of the first two quality assessment exercises (results published in 
Genome Research 9:1-4, 1999; also available at 
http://www.nhgri.nih.gov/Grant_info/Funding/Statements/RFA/quality_standard.html)
 indicated both that evaluation was feasible and that high quality DNA 
sequence data could be generated at a reasonable cost.  Since then, two 
additional exercises have been carried out, with similar results.

These four sequence assessment exercises were conducted by the same 
laboratories that are the primary producers of human sequence data, in a 
'round-robin' manner, in which each center’s data was checked by two other 
centers.  While the ‘round-robin’ format was very useful, both for the NHGRI 
in assessing sequence quality and for the individual centers in monitoring 
their processes, it has several disadvantages for ongoing quality control.  
Since the assessments were done by several centers, there was the possibility 
of a lack of consistency.  The exercises were also time- and resource-
consuming for the sequencing laboratories and, as a result , placed a burden 
on them that undoubtedly reduced their productivity.  With the major increase 
in the rate of data production that will be required to meet the new HGP 
goals, sequence data will have to be assessed in greater quantity and/or at 
greater frequency than in the past.  Continuing the round-robin format would 
thus impose an increasing burden on the sequence production centers.  Finally, 
groups that were, in effect, competing with each other for resources conducted 
the previous exercises. This is less than desirable within the new NHGRI 
management structure for the human and mouse sequencing programs, which relies 
on a high degree of cooperation among the participating centers for their 
success.  

A dedicated Quality Assessment Center (QAC) will address all of these issues.  
In addition, a dedicated QAC will also be able to advance the methods used to 
assess sequence quality as this rapidly moving field progresses.

The recent introduction of an intermediate goal for the HGP of "working draft" 
quality sequence has also had ramifications for sequence data quality 
assessment, and NHGRI believes that a  QAC will also be useful for checking 
working draft data.  The current provisional standard for working draft data 
is that the number of phred 20 bases divided by the total project length 
(calculated both as a sum of fingerprint fragments and as a sum of sequence 
contigs over 1 kb) should be no lower than 3 for any individual project, and 
at least 4 as an average of all a producer’s working draft projects. Since 
production of working draft sequence is a recent endeavor, these standards may 
be re-evaluated.  It is anticipated that the quality of working draft can be 
adequately assessed against this standard by purely computational methods.

In September 1999, NHGRI began a new initiative to sequence the mouse genome. 
Nine groups were funded for sequence production; five of these were new groups 
that are not part of the Human Sequencing Research Network, bringing the total 
number of NHGRI-funded genome sequence production groups to 12. The Mouse 
Sequencing Research Network will also produce both working draft and finished 
sequence. The precise quality standard for mouse genomic sequence is still a 
matter of community discussion, but it is likely that the working draft 
standard and the finished sequence single-base error rate will be similar to 
that for human. 
 
Research objectives and scope

Applications are sought to support one cooperative agreement for a Quality 
Assessment Center (QAC) to evaluate, on an ongoing basis, the quality of both 
the finished and the working draft DNA sequence data being produced by the 
NHGRI-funded human and mouse sequence production centers.  NHGRI expects that 
the QAC will encompass three principal activities: 1) routine quality 
assessment exercises on finished and working draft sequence produced by NHGRI-
funded large-scale sequencers; 2) regular reports of results to the public and 
to NHGRI; and 3) coordination with the internal quality control programs at 
the sequence production centers.   NHGRI is willing to support a fourth 
activity, research on questions about sequence quality, if funds are 
available.

1.  Quality assessment
a. Finished data
The most important activity of the QAC will be the routine assessment of the 
quality of finished and working draft data produced by the NHGRI-funded human 
and mouse sequence production centers.  During the pilot project period, 
several sequence quality assessment exercises were completed; a description of 
the methods used is available at 
http://www.nhgri.nih.gov/Grant_info/Funding/Statements/RFA/quality_standard.html.
 The methods proposed by an applicant responding to this RFA need not be 
identical to those used in the previous exercises.  However, any new methods 
proposed must be robust, well-justified, and be an improvement over existing 
methods.  In any QAC proposal, the sampling methods (e.g., schedule of quality 
assessments, how much material will be sampled) should be addressed.  

In order to perform an effective quality assessment, t is expected that the 
QAC will stay abreast of developments and standards in large-scale genomic 
sequencing and incorporate these advances into a QA program. The QAC should be 
able to use all basecalling and data assembly programs that are used by the 
sequence production centers.  Moreover, the software used by the QAC for 
checking sequence quality should be compatible with that used by the 
sequencing centers to ensure that the file formats used by the sequence 
production centers are readily and reliably transferable to the QAC, and to 
ensure that results are comparable between sequence production centers. 

The NHGRI expects that the data from the sequence production centers will be 
checked on at least a semi-annual basis, and that first assessment of the 
quality of finished sequence should be completed no later than four months 
after funding of the QAC. Applicants should propose a plan that addresses this 
schedule or, with appropriate justification, propose a plan that involves a 
better schedule.  In any particular round of evaluation, the emphasis should 
be on data that has been produced in the interval since the most recent 
previous assessment.  

The raw sequencing data and the bacterial isolates containing large insert 
clones corresponding to the genomic sequence are maintained by the individual 
sequencing centers.  Insofar as these materials will be required for the QAC 
to perform a quality assessment exercise, NHGRI will be responsible for 
ensuring that they are provided to the QAC.  It is expected that the QAC will 
maintain adequate communication with NHGRI and the sequencing centers so that 
the logistical aspects of the QA program will be accomplished smoothly.

b. Working draft data
Applicants responding to this RFA should also propose methods to efficiently 
assess the quality of working draft sequence from each production center. For 
assessment of working draft sequence, a schedule and sampling protocols 
appropriate to the amount of data produced should be proposed.

For the human sequence, the NHGRI-funded sequence production centers will 
produce most of the working draft sequence as soon as Spring 2000 (they will 
also produce a smaller amount of finished sequence during this time).  The 
rate of conversion of the working draft to finished sequence will then 
increase very rapidly in order to meet the NHGRI-DOE five-year goals.  At the 
same time, production of working draft and finished mouse sequencing will be 
increasing during 2000 and 2001.  Applicants should take this anticipated 
schedule into account, including the increasing demand for checking capacity, 
and propose a distribution between the assessment of working draft and of 
finished sequence.  At present, there are three larger, and four smaller, 
NHGRI-funded human sequencing centers, and four new mouse sequencing centers; 
most are producing both draft and finished sequence. 

Although the NHGRI will not provide support for checking data from centers 
funded by other agencies, it is not unreasonable to expect that there will be 
a need for the QAC to assess data from at least two other large sequence 
production centers (the Sanger Centre and the Joint Genome Institute).  
Opportunities for assessment of other large-scale sequence products may arise 
in the future. Ideally, the QAC should have sufficient eventual capacity to 
assess this data as well. However, support for the assessment of data from 
non-NHGRI centers must be negotiated separately. 

2. Reporting

The primary purpose of the regular quality assessment exercises is for the QAC 
to gather data that will enable it to provide the public, the production 
centers and NHGRI  with information about the quality of the sequence data 
being produced. . It is anticipated that the QAC will publish significant 
findings, based on aggregate quality assessment results, about the quality of 
the human genome sequence in the appropriate peer-reviewed literature, within 
the terms and conditions of the Cooperative Agreement. In addition, as part of 
the large-scale sequencing research network, the QAC is also expected to 
provide reports to each of the sequence production centers and to NHGRI about 
the results of quality assessments.  Information about the quality of the 
sequence gathered by the QAC under Item 1 above must be handled equitably, 
expeditiously, and with discretion by the QAC.  The details of the elements of 
these reports will be negotiated with the successful applicant after the award 
is made. However, reports from previous QA exercises have identified the 
following elements as being informative:

 o    the procedures used to perform the assessment, including an indication 
of statistical significance of the result;
o     the general quality of underlying sequence data (read lengths, quality 
of reads, etc.);
o      results pertaining to the verification of assembly (e.g. restriction 
analysis, PCR, computational reassembly results); the likely origin of any 
misassemblies;
o     the number, location and type of discrepancies between the QAC's 
assembly and the GenBank accession;
o     how discrepancies were resolved by the QAC;
o     the number of gaps and ambiguities, and (if readily resolvable) whether 
and how these were resolved;  
o     the overall analysis of quality of finished sequence, including a brief 
narrative assessing the quality of the finished sequence, strengths, 
weaknesses, and an assessment of whether the overall sequence meets the 
community standards; and
o     recommendations to the sequence production center to improve any 
deficiencies.

Applicants should propose any additional items that they believe should be 
incorporated into the quality assessment report.
 
A report format will also be developed for working draft data, in consultation 
with the QAC and NHGRI staff. 

In addition to the report on the individual assessments, it is expected that 
the QAC will periodically report on the aggregate quality assessment data 
obtained from all sequence producers.  Such a report could contain, for 
example, a brief summary of the results from each sequencing production center 
(anonymously), a summary of the aggregate data, comparisons to previous 
aggregate analysis reports, a summary of the kinds of errors or weaknesses in 
the data that were revealed, analysis of the aggregate data that may reveal 
trends of error, sources of error, etc., recommendations to improve the 
quality of the sequence produced by the sequence production centers assessed, 
and recommendations for improving the methods for data quality assessment.  
The aggregate analyses will likely form the basis for a public report on the 
quality of sequence data produced by the HGP, for example by publication in 
the peer-reviewed scientific literature, within the terms and conditions of 
the Cooperative Agreement award. 

3. Outreach/coordination

 It is expected that the QAC will interact productively with the production 
centers, both to maintain the state of the art and to provide advice about 
sequencing quality. QAC investigators are encouraged to visit the production 
centers at regular intervals to maintain close familiarity with the state of 
the art in genomic DNA sequencing, and support for such travel will be 
allowable. 

The P.I. of the QAC will be expected to participate in meetings of the 
principal investigators of the sequence centers, for both human and mouse 
sequencing.  These will be held up to three times per year.  It is expected 
that the successful applicant will be productively integrated into the 
sequencing research networks for the human and mouse, to be better able to 
provide advice and information about quality of genomic sequence data. 

4. Sequence quality research and development 

It is likely that methods for assessing the quality of genomic sequence will 
evolve rapidly.   Because large-scale genomic sequencing is a relatively new 
activity, few methods have been developed to assess the quality of the data 
produced.  NHGRI encourages applicants to include in their application a 
program of research and development to investigate new methods, or to improve 
existing methods, to measure and characterize the quality of genomic DNA 
sequence data.  For example, there are few methods available to routinely 
assess the fidelity of the deposited sequence to the target genome(s), (i.e., 
to verify long or short-range assemblies, or to assess the frequency of 
deletions or rearrangements in deposited sequence relative to the target 
genome); NHGRI believes that assessing these quality parameters will become 
increasingly important over the next two years.  Other suitable topics for 
research and development include statistical optimization of sampling for 
quality assessment exercises; computerized methods for rapid error detection 
in, and assessment of the quality of genomic data; genomic sequencing error 
types, trends and origins; and novel or improved methods of annotating errors 
in genomic sequence databases.

Incorporation of new or improved methods into routine quality assessment 
should be made in consultation with NHGRI.  If limitations in staffing, space 
or other factors arise, or NHGRI funds are limited, the quality assessment 
activity will take precedence over QA research and development.  It is 
expected that QA research projects will occupy no more than about 15% of the 
budget and effort for the project.
	
SPECIAL REQUIREMENTS FOR COOPERATIVE AGREEMENTS

Terms and conditions of award

The following terms and conditions will be incorporated into the award 
statement and will be provided to the Principal Investigator, as well as the 
appropriate institutional official, at the time of award.  The following 
special terms of award are in addition to, and not in lieu of, otherwise 
applicable OMB  administrative guidelines, HHS grant administration 
regulations at 45 CFR Parts 74 and 92 [Part 92 is applicable when State and 
local Governments are eligible to apply], and other HHS, and NIH grant 
administration policies:

1.  Mechanism
The administrative and funding instrument used for this program will be the 
Cooperative Agreement (U01), an "assistance" mechanism (rather than an 
"acquisition" mechanism), in which substantial NIH scientific and/or 
programmatic involvement with the awardee is anticipated during the 
performance of the  activity.  Under the Cooperative Agreement, the NIH 
purpose is to support and/or stimulate the recipient's activity by involvement 
in and otherwise working jointly with the award recipient in a partner role, 
but it is not to assume direction, prime responsibility, or a dominant role in 
the activity.  Consistent with this concept, the dominant role and prime 
responsibility for the activity resides with the awardee(s) for the project as 
a whole, although specific tasks and activities in carrying out the study will 
be shared among the awardee(s) and the NHGRI Program Director.

2.  PI Rights and Responsibilities
The PI will have the primary responsibility for defining the details for the 
project within the guidelines of the RFA and for performing the scientific 
activity.  The PI will agree to accept close coordination, cooperation, and 
participation of NHGRI staff in those aspects of scientific and technical  
management of the project as described under "NHGRI Program Staff 
Responsibilities".

The PI of the quality control center will:

 o     Determine experimental approaches, design protocols, and conduct 
quality assessment of the genomic sequence produced by the sequence production 
centers, 
o      Conduct regular quality assessments of finished and working draft data 
produced by the NHGRI-funded sequence production centers, sufficient to allow 
a reliable and independent assessment of the data quality, 
o     Coordinate with NHGRI and respond to sequencing center P.I. comments 
about the results of individual quality assessments; publish or otherwise 
publicize results of individual quality assessments with the agreement of the 
sequencing production centers and the NHGRI,
o     Publish significant findings on the quality of the aggregate genomic 
sequence,
o     Submit periodic progress reports in a standard format, as agreed upon by 
the scientific consultants to the large-scale sequencing program,
o     Accept and implement the common guidelines and procedures approved by 
the scientific consultants to the large-scale sequencing program and NHGRI, 
including future decisions about sequence quality standards, and
o    Attend meetings of the scientific consultants to the large-scale 
sequencing program.

3.  NHGRI Program Staff Responsibilities:

The NHGRI Program Director will have substantial scientific/programmatic 
involvement during the conduct of this activity through technical assistance, 
advice and coordination such as participating in the design of the collective 
activities of the sequence production centers, advising in the selection of 
sources or resources, coordinating or participating in collection and/or 
analysis of data, advising in management and technical performance, or 
participating in the preparation of publications.  However, the role of NHGRI 
will be to facilitate and not to direct the activities.  It is anticipated 
that decisions in all activities will be reached by consensus of the sequence 
production centers and that NHGRI staff will be given the opportunity to offer 
input to this process. This substantial involvement is in addition to normal 
stewardship provided for the award by NHGRI.

The Program Director will:

o      Participate in the group process-setting research priorities, deciding 
optimal research approaches and protocol designs, and contributing to the 
adjustment of research protocols or approaches as warranted.  The Program 
Director will assist and facilitate the group process and not direct it,
o      Serve as liaison, helping to coordinate activities among the members of 
the scientific consultants to the large-scale sequencing program, act as a 
liaison to the NHGRI, and as an information resource about extramural genome 
research activities,
o      Attend the meetings of the scientific consultants to the large-scale 
sequencing program, assist in developing operating guidelines, quality control 
procedures, and consistent policies for dealing with recurrent situations that 
require coordinated action.  The Program Director must be informed of all 
major interactions of the scientific consultants to the large-scale sequencing 
program.  The NHGRI Program Director will be responsible for scheduling the 
time and preparing concise (3 to 4 pages) minutes or a summary of the meetings 
of the scientific consultants to the large-scale sequencing program, which 
will be delivered to members of the group within 30 days after each meeting,
o      Lend his/her relevant expertise and overall knowledge of the NHGRI- and 
NIH- sponsored research to facilitate the selection of scientists not 
affiliated with the awardee institutions who are to serve on the scientific 
consultants to the large-scale sequencing program,
o     Serve on subcommittees of the scientific consultants to the large-scale 
sequencing program as appropriate,
o      Provide advice in the management and technical performance of the 
investigation,
o      Serve as scientific liaison between the awardees and other program 
staff at NHGRI,
o      Assist in promoting the availability of the human genome sequence and 
related resources to the scientific community at large,
o      Retain the option to recommend the withholding or reduction of support 
if the QAC fails to carry out routine QA exercises in such a way as to ensure 
that the quality of sequence produced by the production centers is adequately 
monitored, and
o      Participate in data analyses, interpretations, and where warranted, co-
authorship of the publication of results of studies conducted by the QAC.
 
4. Scientific consultants to the large-scale sequencing program

The scientific consultants to the large-scale sequencing program are 
responsible for discussing progress within the collective organization of the 
sequence production centers funded under RFA HG-98-002 and RFA HG-98-003, and 
for advising NHGRI as to how the human and mouse DNA sequence can be completed 
within the stated goals of time and accuracy, and within budget.  As part of 
their role in advising NHGRI, the scientific consultants will provide counsel 
to the NHGRI about developing uniform procedures for data quality assessment.  
The scientific consultants will be senior scientists with relevant expertise.  
The Director, NHGRI, will select the scientific consultants.

The scientific consultants will meet as a group at least once a year.  The 
first part of this meeting will be joint with the PI’s of the sequence 
production centers and the QAC.  Annually, the scientific consultants will 
make recommendations regarding progress of the QAC, along with recommendations 
about progress at the sequence production centers, and offer recommendations 
to the Director, NHGRI about any changes which may be necessary in the 
program.

5.  Arbitration Process

Any disagreement that may arise on scientific/programmatic matters (within the 
scope of the award), between the award recipient and the NHGRI may be brought 
to arbitration.  An Arbitration Panel, composed of three members - one QAC 
designee, one NHGRI designee, and a third designee with expertise in the 
relevant area and chosen by the other two designees, will be convened.  This 
special arbitration procedure in no way affects the awardee's right to appeal 
an adverse action that is otherwise appealable in accordance with PHS 
regulations 42 CFR Part 50, Subpart D and HHS regulation at 45 CFR Part 16.

6.  Yearly Milestones

The awardee will be asked to define yearly milestones, including information 
about expected progress on sequence quality research and development, at the 
time of the award and to adjust these milestones annually at the anniversary 
date.  In accord with the procedures described above, NHGRI may withhold or 
reduce funds if the QAC substantially fails to meet its milestones or to 
maintain the center at the state of the art. A mandatory yearly milestone for 
the QAC is that regular QA exercises will be carried out on all NHGRI-funded 
sequencing production centers, sufficient to constitute an independent 
evaluation of the quality of finished and working draft data. At a minimum, 
the data must be assessed from each center twice per year. Failure to meet 
this milestone will result in a programmatic finding that progress is 
inadequate. 

LETTER OF INTENT

Prospective applicants are asked to submit a letter of intent that includes a 
descriptive title of the proposed research, the name, address, and telephone 
number of the Principal Investigator, the identities of other key personnel 
and participating institutions, and the number and title of the RFA in  
response to which the application may be submitted. Although a letter of 
intent is not required, is not binding, and does not enter into the review of 
a subsequent application, the information that it contains allows IC staff to 
estimate the potential review workload and avoid conflict of interest in the 
review.

The letter of intent is to be sent to the program staff listed under INQUIRIES 
by the letter of intent receipt date listed in the heading of this RFA.

APPLICATION PROCEDURES

Applications are to be submitted on the grant application form PHS 398 (rev. 
4/98).  These forms are available at most institutional offices of sponsored 
research; from the Division of Extramural Outreach and Information Resources, 
National Institutes of Health, 6701 Rockledge Drive, MSC 7910, Bethesda, MD 
20892-7910, telephone 301/435-0714, email: grantsinfo@nih.gov; and on the 
internet at http://grants.nih.gov/grants/funding/phs398/phs398.html and from 
the program administrator listed under INQUIRIES. Because of the multi-
component scope of the RFA, applicants will likely require more than the usual 
space allowed by the page limits for the Research Plan described in form PHS 
398 to address the four components described in the Research Objectives and 
Scope above. Therefore, the page limit for the application Research Plan for 
the first three components taken together (Quality assessment, Reporting, and 
Outreach/Coordination) is 25 pages.  If a proposal for sequence quality 
research and development is included, an additional 25 pages will be allowed 
in the application Research Plan for this fourth component.

The RFA label available in the PHS 398 (rev. 4/98) application form must be 
affixed to the bottom of the face page of the application.  Failure to use 
this label could result in delayed processing of the application such that it 
may not reach the review committee in time for review.  The RFA number must be 
on the label.  In addition, the RFA title and number must be typed on line 2 
of the face page of the application form and the YES box must be marked.

The sample RFA label is available at: 
http://grants.nih.gov/grants/funding/phs398/label-bk.pdf has been modified to 
allow for this change.  Please note this is in pdf format.

Submit a signed, typewritten original of the application, including the 
Checklist, and three signed photocopies, in one package to:

CENTER FOR SCIENTIFIC REVIEW
NATIONAL INSTITUTES OF HEALTH
6701 ROCKLEDGE DRIVE, SUITE 1040 - MSC 7710
BETHESDA, MD  20892-7710
BETHESDA, MD  20817 (for express/courier service)

At the time of submission, two additional copies of the application must also 
be sent to:

Dr. Rudy Pozzatti
Office of Scientific Review
National Human Genome Research Institute
Building 31, Room B2B37, MSC 6050
Bethesda, MD  20982-6050
Telephone:  (301) 402-0838

Applications must be received by March 14, 2000.  If an application is 
received after that date, it will be returned to the applicant without review.  
The Center for Scientific Review (CSR) will not accept any application in 
response to this RFA that is essentially the same as one currently pending 
initial review, unless the applicant withdraws the pending application.  The 
CSR will not accept any application that is essentially the same as one 
already reviewed. This does not preclude the submission of substantial 
revisions of applications already  reviewed, but such applications must 
include an introduction addressing the previous critique. 
Applicants must consider and address the following in preparing an application 
for the QAC:

General. The applicant must be able to carry out an independent assessment of 
the sequence data produced by the production centers. This does not preclude 
the QAC from being in the same institution as an NHGRI-funded production 
center.  In such a situation, however, it must be clear that the QAC will be 
completely independent of the production center and that no personnel of the 
QAC are funded in any part by a grant that funds a production center.

QA exercise plan. Applicants should propose and justify a method for carrying 
out the routine assessments of data produced by the human and mouse production 
centers.  The proposal must include a plan, including a schedule, to 
accomplish these assessments in a timely manner. The applicant must also 
provide sufficient information to justify the ability of the plan and 
personnel to meet the organizational challenges of conducting routine quality 
assessments on the scale requested in this RFA.  Applications that can 
demonstrate cost efficiencies in carrying out the QA exercises (for example, 
from approach, scale, automation, organization, etc.) will be at an advantage.  
Applications that do not adequately address the QA exercise component for 
finished and working draft sequence will be considered non-responsive to this 
RFA. 

Applications should address the potential for growth in capacity of the QAC to 
be able to account for growth in the production of sequence data by NHGRI-
funded sequence production centers, as outlined above.

Outreach/coordination. It is extremely important that the QAC be able to 
interact in a positive and productive way with the other participants in the 
Research Network. The information generated by the QAC will potentially be 
made public, used in making funding decisions and remedying quality concerns, 
should they arise, in the sequence production centers. Applicants should 
provide information that indicates their willingness and ability to integrate 
successfully and productively with the cooperative organization of the 
sequence production centers. 
	
Sequence quality research. Applicants who choose to request funds for this 
component should provide a detailed research plan for the entire grant period. 
Examples of areas suitable for sequence quality assessment research include 
(but are not limited to) methods to assess fidelity of the deposited sequence 
to the target genome(s); statistical optimization of sampling for Quality 
Assessment exercises; computerized methods for rapid error detection in, and 
assessment of the quality of genomic data; genomic sequencing error types, 
trends and origins; novel or improved methods of annotating errors in genomic 
sequence databases. It is anticipated that successful methods developed in the 
research component will be applied to routine quality assessment exercises as 
appropriate.  Applicants should justify the utility and relevance of the 
proposed research, in addition to addressing novelty and feasibility. 

REVIEW CONSIDERATIONS

A. General

Upon receipt, applications will be reviewed for completeness by CSR and for 
responsiveness by the NHGRI.  Incomplete applications will be returned to the 
applicant without further consideration.  If NHGRI staff find that the 
application is not responsive to the RFA, it will be returned without further 
consideration.

Applications that are complete and responsive to the RFA will be evaluated for 
scientific and technical merit by an appropriate peer review group convened by 
the (IC) in accordance with the review criteria stated below. As part of the 
initial merit review, a process will be used by the initial review group in 
which applications receive a written critique and undergo a process in which 
only those applications deemed to have the highest scientific merit, generally 
the top half of the applications under review, will be discussed, assigned a 
priority score, and receive a second level review by the (IC) National 
Advisory Council or Board. The second level of review will be provided by the 
National Advisory Council for Human Genome Research.

All applications will be judged on the basis of the scientific and technical 
merit of the proposed projects and the documented ability of the investigators 
to meet the RESEARCH OBJECTIVES of the RFA.

B. Review criteria
 
In the written comments, reviewers will be asked to discuss the following 
aspects of the application in order to judge the likelihood that the proposed 
work will accomplish the goals of this RFA. Each of these criteria will be 
addressed and considered in assigning the overall score, weighting them as 
appropriate for each application.

For the Quality assessment, Reporting, and Outreach/Coordination components:
o      Quality of the plan to assess the quality of the finished DNA sequence 
being produced by the production sequencing centers. 
o        Quality of the plan to assess whether working draft sequence data 
produced by the production sequencing centers meets the NHGRI standards.
o       Ability of the PI to integrate productively with the collective 
organization of the sequence production centers.
o       Track Record of the PI and other key personnel.
o      Quality of the management plan, including organization, integration and 
allocation of personnel and equipment, and plans for attaining cost 
efficiencies.
o      Availability of the facilities, resources, expertise and technology 
necessary to perform the research, and the level of institutional commitment.
o     Appropriateness of the proposed budget, schedule and time-line in 
relation to the proposed quality assessment program.

For the Sequence Quality Research and Development component: 

Quality of the proposed plan, if any, for sequence quality research and 
development; including significance, probability of success, innovation of 
approach, and relevance to future genomic sequencing QA needs.

1.  Significance: Does this study address an important problem? If the aims of 
the application are achieved, how will scientific knowledge be advanced? 
What will be the effect of these studies on the concepts or methods that 
drive this field? Is the proposed work relevant to the current and 
projected needs for assessing quality of genomic sequence produced at large 
scale?

2. Approach:  Are the conceptual framework, design (including composition of 
study population), methods, and analyses adequately developed, well 
integrated, and appropriate to the aims of the project? Does the applicant 
acknowledge potential problem areas and consider alternative tactics?


3.   Innovation: Does the project employ novel concepts, approaches or 
methods? Are the aims original and innovative? Does the project challenge 
existing paradigms or develop new methodologies or technologies?

4.   Investigator:  Is the investigator appropriately trained and well suited 
to carry out this work? Is the work proposed appropriate to the experience 
level of the principal investigator and other researchers (if any)? PLEASE 
DO NOT INCLUDE descriptive biographical information unless important to the 
evaluation of merit.

5. Environment: Does the scientific environment in which the work will be done 
contribute to the probability of success? Do the proposed experiments take 
advantage of unique features of the scientific environment or employ useful 
collaborative arrangements? Is there evidence of institutional support?  

In addition to the above criteria, in accordance with NIH policy, all 
applications will also be reviewed with respect to the following: 

o    The reasonableness of the proposed budget and duration in relation to the 
proposed research 
o    The adequacy of the proposed protection for humans, animals or the 
environment, to the extent they may be adversely affected by the project 
proposed in the application. 
o     The initial review group will also examine the provisions for the 
protection of human subjects and the safety of the research environment. 

The second level review will be conducted by the National Advisory Council for 
Human Genome Research.

AWARD CRITERIA
Awards will be made on the basis of scientific and technical merit as 
determined by peer review and the availability of funds. 

INQUIRIES
Written and telephone inquiries concerning this RFA are encouraged. The 
opportunity to clarify issues or questions from potential applicants is 
welcome.

Direct inquiries regarding programmatic issues to:

Dr. Adam Felsenfeld
Division of Extramural Research
National Human Genome Research Institute
Building 31, Room B2B07, MSC 6050
Bethesda, MD  20892-6050
Telephone:  (301) 496-7531
FAX:  (301) 480-2770
Email:  Adam_Felsenfeld@nih.gov

Direct inquiries regarding fiscal matters to:

Ms. Jean Cahill
Grants Management Office
National Human Genome Research Institute
Building 38A, Room B2B34 , MSC 6050
Bethesda, MD  20892-6050
Telephone:  (301) 402-0733
FAX:  (301) 402-1951
Email:  Jean_Cahill@nih.gov

AUTHORITY AND REGULATIONS

This program is described in the catalog of Federal Domestic Assistance No. 
93.172.  Awards are made under the authority of the Public Health Service Act, 
Title IV, Part A (Public Law 78-410, as amended by Public Law 99-158, 42 USC 
241 and 285) and administered under PHS grants policies and Federal 
Regulations 42 CFR Part 52 and 45 CFR Parts 74 and 92.  This program is not 
subject to the intergovernmental review requirements of Executive Order 122372 
or Health Systems Agency review.

The PHS strongly encourages all grant and contract recipients to provide a 
smoke- free workplace and promote the non-use of all tobacco products.  In 
addition, Public Law 103-227, the Pro-Children Act of 1994, prohibits smoking 
in certain facilities (or in some cases, any portion of a facility) in which 
regular or routine education, library, day care, health care or early 
childhood development services are provided to children.  This is consistent 
with the PHS mission to protect and advance the physical and mental health of 
the American people.

Return to Volume Index

Return to NIH Guide Main Index

	Office of Extramural Research (OER)			National Institutes of Health (NIH) 9000 Rockville Pike Bethesda, Maryland 20892			Department of Health and Human Services (HHS)
Note: For help accessing PDF, RTF, MS Word, Excel, PowerPoint, RealPlayer, Video or Flash files, see Help Downloading Files.