FDA Logo links to FDA home page
U.S. Food and Drug Administration
HHS Logo links to Department of Health and Human Services website

FDA Home Page | Search FDA Site | FDA A-Z Index | Contact FDA

horizonal rule

Department of Health and Human Services
U.S. Food and Drug Administration
Congressional Justification
FY 2004 Annual Performance Plan
FY 2003 Revised Final Performance Plan
FY 2002 Annual Performance Report

Appendix E: Verification and Validation


Agency Wide

FDA will ensure consistency in the tracking and reporting of the administrative management performance goals. In addition, FDA is taking steps to routinely monitor this data and take appropriate actions as needed. Data is from a variety of sources for these performance goals including the Annual Chief Financial Officer's Report, Civilian and Commission Corps personnel databases, monthly and annual full-time equivalent (FTE) reports and data-runs, the FDA FAIR Act Inventory and the FY 2001 FDA Workforce Restructuring Plan, monthly statements from bank card companies and the FDA Small Purchase System.

Foods

Public health data systems currently are not adequate to provide accurate and comprehensive baseline data needed to draw direct relationships between FDA's regulatory activities and changes in the number and types of foodborne illnesses that occur annually in this country. Because of the need to have better data on food related illnesses, FDA and USDA began working with CDC in 1995 to improve food safety surveillance. FoodNet, an active surveillance program, was created through this joint effort. Currently there are nine FoodNet sites.

These sites, which operate in areas that are representative of the geographic and demographic population distributions in this country, provide much better data on the number of foodborne illnesses and trends in terms of the types of contaminants that are causing these illnesses. This type of information can be critical to efforts by food safety agencies to redirect their regulatory and research resources to those food safety problems that pose the greatest threat to the health of consumers. Moreover, in 2002 when the data will be sufficient in volume and quality to establish baselines against which to measure changes in foodborne illnesses, FDA will be in a better position to establish broad scope outcome goals that are essential to effective performance planning.

Food Safety regulation development and research activities are planned and tracked through internal management systems. Progress on the development of regulations is tracked mainly through CFSAN's document tracking system and the Federal Register document tracking system. These systems permit the Agency to track the processing of regulations from the time they are filed to the point at which action is complete-usually the publication of a final regulation in the Federal Register.

CFSAN uses a number of internal data systems to track premarket review progress. These include the Management Assignment Tracking System (MATS) to track progress of petition reviews, Correspondence Tracking System (CTS) to track progress on biotechnology consultations, reviews of GRAS notifications, nutrient content claims, and health claims petitions/notifications. Outcome-oriented performance information can be extracted from MATS only by a labor-intensive manual process. CFSAN's internal data systems are limited to tracking time to a completed review and do not have the capability to track distinct phases of the review process. In FY 1998, the Office of Premarket Approval's (OPA) internal database was modified to permit more detailed tracking of CFSAN's action on biotechnology consultations. In FY 1999, CFSAN implemented an electronic workflow system that will replace MATS and CTS and permit real-time monitoring of review progress. The electronic workflow system is expected to be in full use in FY 2001. The new system will track automatically actions related to the processing of food and color additive petitions, GRAS petitions and biotechnology consultations.

FDA uses a variety of data systems to develop and verify performance goals for its food safety activities. Among these are several field data systems. The most important of the field data systems are the Program Oriented Data System (PODS) and the Operational Administrative System for Imports (OASIS). PODS tracks field activities conducted by FDA's field force and the firms over which FDA has legal responsibility. Information provided by this system includes data on the number of inspections, wharf examinations, sample collections and analyses as well as the time spent on each. OASIS, which is coordinated with the U.S. Customs Service, provides data on what products are being imported as well as where they are arriving. It also provides information on compliance actions related to imports. In FY 2001, the Field Accomplishments Tracking System (FACTS) will be the primary mechanism for tracking compliance activities for the domestic food industry. The National Seafood HACCP Compliance Database System maintains information on seafood HACCP inspections conducted by FDA and states in partnership with FDA. Standardized forms (Cardiff forms) assure comparability of HACCP compliance data whether FDA or states conduct the inspections. Another field data collection instrument is the field survey. Field surveys are special assignments that are developed and implemented specifically to collect information needed to more thoroughly evaluate the nature and extent of particular postmarket food safety problems.

Data are also gathered through a number of other surveys designed for specific purposes. These include the Health and Diet Survey that provides information required to evaluate the impact of the Agency's food labeling activities. These surveys include questions that are designed to query consumers on how they use food labeling information to make decisions to use or purchase food products. Another survey is the NASS survey currently being developed jointly by FDA and USDA to evaluate the impact of GAPs and GMPs for improving the safety of fresh fruits and vegetables. The survey questions will be designed to provide data on practices employed in the production and processing of fresh fruits and vegetables. The results of the NASS surveys will be used to establish baselines for industry practices as well as evaluate the impact of voluntary GAPs and GMPs on improving production and processing practices for fresh produce.

Comprehensive data on illness caused by food and cosmetic products is critical to efforts to protect the health of consumers. Some of the illness data are provided by databases that contain information on adverse events, reported by consumers and industry on food and cosmetic products. In FY 2001, the Agency began improving the quality and accessibility of data on adverse events through the development and implementation of a new adverse event reporting system for dietary supplements. In FY 2002, the Agency will build upon the system nodule for dietary supplements by developing and implementing an integrated adverse reporting system for all food and cosmetic products.

Proposed research projects are subjected to management reviews prior to implementation and periodic management reviews after the projects have been initiated. The primary planning and management system for food safety research is the Center Program Resources (CPR) plan system that provides quarterly resource use reports and semi-annual reports on accomplishments versus planned milestones. In FY 2000, the Center formed a research management task group responsible for evaluating related processes and systems and developing recommendations for improvement. In addition, research projects are subjected to periodic external peer reviews. Peer reviews by recognized scientific experts in various disciplines related to food safety provide objective feedback that helps FDA evaluate the progress, quality and relevance of its research activities. In addition, risk assessment models are verified periodically using statistical models that assess their ability to make rapid and accurate estimates of risks associated with a particular food safety hazard.

In FY 1999, the Center began implementation of its Resource Planning, Prioritization, and Allocation Process. The primary purpose of this Process is to provide pertinent data throughout the fiscal year on program activities, including GPRA performance goals, Center program priorities, Congressional directives, statutory responsibilities under FDAMA, and Food Safety Initiative objectives.

Drugs

A preliminary assessment for data completeness, accuracy, and consistency and related quality control practices was done for each performance goal. The purpose of the assessment was to determine if the data was of a sufficient quality to document performance and report program results, whether the data was appropriate for the performance measure and if it was considered sound and convincing. The Center obtained from its programs a description of the means that are used to verify and validate measured values for each performance goal. CDER has a number of quality control processes in place to ensure that performance data is reliable. Below are descriptions of several data systems used by CDER.

Adverse Event Reporting System (AERS)

The Adverse Event Reporting System (AERS) is an Oracle based computerized information system designed to support the Agency's post-marketing safety surveillance program for all approved drug and therapeutic biologic products. The structure of the database is in compliance with the international safety reporting guidance (ICH E2B), including content and format for electronic submission of the reports from the manufacturers. Features include on-screen review of reports, searching tools, and various output reports in support of postmarketing drug surveillance and compliance activities. The ultimate goal of AERS is to improve the public health by providing the best available tools for storing and analyzing safety reports.

Currently, reports are received either on paper as MedWatch forms or electronically. AERS assigns an individual safety report (ISR) identification number for each report. Paper submissions are scanned and stored in retrieval software. All data elements are entered and undergo data entry quality control to ensure completeness and accuracy. All reported adverse event terms are coded into a standardized international terminology, MedDRA (the Medical Dictionary for Regulatory Activities). This process is also subjected to coding quality control. After data entry, the reports are routed directly to assigned clinical reviewers in the postmarketing office. The reports are assessed individually and in aggregate for safety concerns.

The functions and tools developed in AERS provide the ability to easily customize queries; such queries are performed by multiple users on a daily basis for any drug and/or adverse event of interest. Standardized report outputs from AERS provide useful postmarketing information to many users within and outside FDA. These functions, combined with appropriate management and processes developed by the FDA, make AERS an effective tool for pharmacovigilance. There is an ongoing process in place to further improve the performance and functionality of AERS. Because pharmacovigilance is a constantly changing field and the volume of postmarketing safety information continues to increase annually, AERS will need modifications and improvements to maintain its usefulness to the FDA users.

AERS was designed to allow for electronic submission of individual case safety reports. Electronic submissions provide CDER, FDA, and the public with several tangible benefits. Specifically, automating the receipt and processing of safety reports will allow CDER to be more responsive to public health issues, greatly reduce resources associated with data management, and apply better data and better science to the drug regulatory process.

However, there are FDA regulatory and infrastructure changes needed for full-scale implementation of electronic submissions. The full-scale implementation requires CDER to develop processes for both electronic data management and pharmacovigilance. Accordingly, CDER has proposed a step-level implementation that will allow CDER to identify and resolve several process issues while the regulatory and infrastructure changes are implemented. This step-level implementation includes a pilot program. This program allows CDER to work with manufacturers who voluntarily submit safety reports electronically. Besides AERS resources being used for the users, AERS resources are used for this pilot program to work with the manufacturers for the implementation of the electronic submissions program of the safety reports. In conjunction with the pilot, proposed rulemaking is being written to require that manufacturers submit suspected adverse drug reaction reports electronically.

As we gain more experience with the pilot electronic submissions program with the manufacturers, maintenance and improvements will be needed to make it more functional and successful. AERS was designed to accommodate electronic submission of adverse event reports from the manufacturers based on ICH specifications. Periodically, these specifications are modified and updated. Therefore some of the AERS maintenance will be due to changing ICH specifications. For example, currently, there is a new version that needs to be implemented. The manufacturers' participation in the pilot program is delayed until the new version is in place. This maintenance also includes MedDRA version upgrades in AERS. This is to assure that the electronic submissions utilizing the current version of MedDRA from the manufacturers are compatible with the version utilized in AERS.

The ultimate goal of the electronic submissions program is to be able to exchange safety reports with other regulators and manufacturers. Currently, we are only able to receive reports electronically. Some of the pilot program manufacturers are able to send reports electronically and are working with their affiliates to be able to receive reports too. We need to be able to share and send reports electronically with other regulators and industry.

In summary, the AERS database in the FDA assures that postmarketing adverse event reports are completely and accurately entered, quality controlled and reviewed to monitor product safety and to protect the public health. The data are valid for this goal because they measure the required performance indicator of expediting the process and evaluation of adverse drug events.

Pediatric Exclusivity Database and the Pediatric Page database (Database enhancements required to meet goal)

The Pediatric Exclusivity Database tracks all data regarding pediatric exclusivity as mandated by FDAMA. Specifically, this database tracks the number of Written Requests issued and the number of products for which pediatric studies have been submitted and for which exclusivity determinations have been made.

The document room enters the date on which a Proposed Pediatric Study Request (PPSR) is received and when the Agency issues a Written Request (WR). Then the pediatric team enters the information pertaining to the types of studies to be conducted. Once the final pediatric studies are submitted to the Agency, the document room enters the receipt date into the database. The project manager for the Pediatric Team enters any additional information pertaining to the granting or denial of exclusivity. The data is quality controlled each month by the pediatric team when they complete their monthly statistics update.

The major strength of this database is that it captures all data relative to exclusivity. Maintaining the database is time consuming for the pediatric team, i.e., entering the data on the studies. However, the document room staff are not trained to recognize what types of studies are requested in the WRs so it is not feasible for them to enter this data themselves.

The Pediatric Page Database was redesigned, piloted, and implemented in July 2000. This database was designed to capture data pertaining to the Pediatric Final Rule, i.e., whether or not pediatric studies required under the rule were completed, the number of waivers and deferrals granted, and the age ranges that may be waived, deferred, or have actually been completed. The project managers consult with the medical officers to determine whether pediatric studies are necessary, waived, or deferred and what ages should be included in the study. Then the project manager enters the information into the database. This information must be entered prior to the approval of an NDA or supplement. The pediatric page, with all relevant pediatric data, is then printed from the database and included with the action package. The action package is then forwarded to various people, i.e., the appropriate reviewer, project managers, team leader, deputy division director, division director, and office director (for NDAs only) who verify the pediatric data and sign off on the package.

The previous version of the database required a password and was not user friendly. Therefore, many project managers did not use the system resulting in incomplete data for a number of applications. The database has been updated, no longer requiring a password, and is now web-based. Training has been provided to the divisions on the new version. The number of pediatric patients being requested to be involved in studies and the types of studies being requested are tracked manually and maintained by individuals in separate databases on their computers or on common drives. Alternatives are being considered to make this an electronic process as well.

The Pediatric Inpatient Database is still being negotiated. Once this information is available to the pediatric team it will be able to determine exactly what drugs are being used in the pediatric population for unlabeled indications and then focus on requesting the studies that are necessary in order to get the products properly labeled.

This information demonstrates that the data in the Pediatric Exclusivity Database and the Pediatric Page Database are complete and accurate and that appropriate quality control practices are in place. The data are valid for this goal because they measure the required performance indicators.

Center-wide Oracle Management Information System (COMIS)

The Center-wide ORACLE Management Information System (COMIS) is CDER's enterprise-wide system for supporting premarket and postmarket regulatory activities. It consists of multiple applications, or components, that store and retrieve data in a single integrated database. COMIS is the core database upon which most mission-critical applications are dependent. The new drug evaluation (NDE) and abbreviated new drug application (ANDA) portions of COMIS contain information about investigational new drug applications (INDs), new drug applications (NDAs), abbreviated new drug applications (ANDAs), supplements, and amendments, and it tracks their status throughout the review process. The type of information tracked in COMIS includes status, type of document, review assignments, status for all assigned reviewers, and other pertinent comments.

CDER has in place a quality control process for ensuring the reliability of the performance data in COMIS. Document room task leaders conduct one hundred percent daily quality control of all incoming data done by their IND and NDA technicians. Senior task leaders then conduct a random quality control check of the entered data in COMIS.

The task leader then validates that all data entered into COMIS are correct and crosschecks the information with the original document. Once the data are saved in COMIS, the document room staff no longer have the capability to change certain document fields. If a data entry change is necessary on any restricted field, the task leader or senior task leader must send a written change request to the Records Management Team (RMT), Office of Information Technology (OIT). Once the change has been made, the document room is notified and the senior task leader/task leader rechecks the data for accuracy.

The Records Management Team (RMT) has three Technical Information Specialists (TIS) assigned to the document rooms in Parklawn, Woodmont II, Corporate Boulevard, Metro Park North II and Wilkins Avenue who oversee the daily activities within their building document rooms. Quality control checks are done on application jackets, outgoing letters, memoranda and reviews, procedure and programming changes and all other activities that take place in their document rooms.

Overall, the data in COMIS are complete and accurate, and appropriate quality control practices are in place. A limited number of people in RMT and the Division of Applications Development Services (DADS), OIT, have authority to input data into COMIS, which helps to protect the integrity of the data. Once entered into the system, data are immediately accessible to users.

Meetings are held on a weekly basis to discuss any and all issues related to COMIS data entry, document rooms, and procedure changes to ensure that COMIS reflects changes in policy and legislative requirements. Attendees at these meetings include two members of the Document Control Room contract management staff in RMT, a Chief Project Manager review division representative from Parklawn, WOCII and Corporate Boulevard, a programmer from DADS, and representatives from the Division of Drug Marketing, Advertising, and Communications, the Office of Generic Drugs, and the Reports and Data Management Team, ORM.

The data obtained from COMIS are valid for this goal because they measure the required performance indicators, e.g., the numbers and types of submissions, receipt dates, and review times. Preliminary discussions have taken place to alleviate system weaknesses and redesign the system in phases over the next few years to improve efficiency. These weaknesses include a manual, paper-driven quality control process, inflexibility of the system to reflect policy and legislation changes in a timely manner, slow or unavailable network connections impeding a user's ability to acquire requested data, and unrecognizable codes requiring tracking to be done manually.

Biologics

The Biologics Program uses various databases to manage its diverse programs and to assess performance. The principal CBER database is the Regulatory Management System-Biologics License Application (RMS-BLA). The RMS-BLA is CBER's new VAX-based, Oracle database that is used to track all biologics license applications, and supplement submissions; provide information to facilitate the review process (product, application status, milestone tracking, facility, review committee, industry contacts, and other information); and produce a wide variety of management reports. The RMS-BLA records application review information on each license application and supplement received and filed by the Center. The RMS-BLA records information about PDUFA and non-PDUFA license applications. The milestone tracking module is used to track and report on CBER's PDUFA goals. Data entry is done in each of the offices' application review divisions. The Regulatory Information Management Staff (RIMS) monitors and is responsible for maintaining data quality and integrity in RMS-BLA.

The Biologics Investigational New Drug Management System (BIMS) is CBER's VAX-based, Oracle database that is used to track all Investigational New Drug Applications (IND), Investigational Device Exemption (IDE), and Master Files (MF) submissions (over 12,000 in 1999); provide product, application status, and other information to facilitate the review process; and produce a wide variety of management reports. The system also stores summaries of telephone conversations and meetings related to the submissions, as well as actually generating some of the correspondence to sponsors. Most data entry is done by the Document Control Center (DCC) or by the Consumer Safety Officers in each office's application review division. There are numerous mechanisms established for quality control in DCC, the application review offices, the Regulatory Information Management Staff, and several built into BIMS itself.

The Blood Logging and Tracking System (BLT) was developed by the Office of Blood Research and Review (OBRR) to record and track the various applications reviewed by that Office. The OBRR receives and reviews a wide variety of application types. PLAs, ELAs (Establishment License Applications) and BLAs are tracked by the RMS-BLA, discussed above. INDs are tracked by the BIMS, also discussed above. The Office utilizes the BLT to record and track data concerning device premarket applications (PMAs) and PMA supplements, 510(k)s, and Abbreviated New Drug Application (ANDAs) and ANDA supplements. The Office also has an NDA tracking system.

The data retrieved from these systems are reviewed and validated by the RIMS and the application review offices. If errors are detected, they are corrected.

Federal regulations (21 CFR, Part 600.14 and 606.171) require reporting of deviations in the manufacture of biological products that affect the safety, purity, or potency of the product. The Biological Product Deviation Reports (BPDRs) (previously called error and accident reports) enable the Agency to evaluate and monitor establishments, to provide field staff and establishments with trend analyses of the reported deviations and unexpected events, and to respond appropriately to reported biological product deviations to protect the pubic health. The regulation applies to licensed manufacturers, unlicensed registered blood establishments, and transfusion services which had control over the product when a deviation occurred to report to FDA the biological product deviation if the product has been distributed.

In May 1995, the DHHS Office of the Inspector General issued a report recommending that the reporting requirements be expanded to include unlicensed blood banks and transfusion services. A proposed rule was issued on September 23, 1997, that expands the reporting requirements to all biological product manufacturers regulated by FDA. The final rule was published on November 7, 2000. On August 10, 2001, FDA published two draft guidance documents: (1) "Draft Guidance for Industry: Biological Product Deviation Reporting for Blood and Plasma Establishments," and, (2) "Draft Guidance for Industry: Biological Prodcut Deviation Reporting for Licensed Manufacturers of Biological Products Other than Blood and Blood Components." The comment period for the guidance documents ended November 13, 2001.

In FY 2001, the Agency received 25,367 biologics product deviation reports. FDA estimates that over 27,000 biologic product deviation reports would be received under the proposed regulation. In June 2001, FDA implemented an electronic reporting system to permit the electronic submission of biologic product deviation reports. This will allow the Agency to receive electronic submission of reports; and to process, analyze and evaluate more than 27,000 reports annually.

The Biologics Program relies in the Office of Regulatory Affairs' Field Accomplishments and Tracking System (FACTS) to register and record biologics manufacturing establishment inspection and compliance data. FACTS versions 1 and 2 together will replace the several dozen applications that comprise the current Field Information System (FIS). The software development contractor delivered FACTS version 1 to the FDA on September 30, 1997. Version 1 functionality includes all sample collections; all sample tracking, accountability, and dispositions; sample analysis of pesticides, additives, colors, elements, mycotoxins and radionuclides; firms inventory, maintenance and registration; work assignments and work management; and other features.

Meanwhile, the design and development of FACTS version 2 is underway. Major features of version 2 include replacing the remaining FIS functions: remainder of lab analyses; inspections; rest of investigations including records and tracking; compliance functions; other core items including personnel management (MUS); and miscellaneous operations including recalls and audit checks.

Animal Drugs and Feeds

An integral part of the FDA continual improvement initiative has been upgrading our data processing and information systems. This includes automation of manual systems and integration of existing systems, which reduces duplication and chances of data entry errors. Our information and data collection systems contain automatic data checks such as comparisons against lists of "valid" responses for a given data field. By programming "business rules" into our systems, the chance for "human" error is reduced. For example, due dates for applications are appropriately assigned and review time is accurately tracked. Data access is restricted to ensure that only appropriate personnel can enter data, review data, or audit the data; checks are in place to ensure that the person who enters the data does not audit the data.

CVM works with data from other governmental agencies such as CDC and USDA. To ensure that our federal partners address our data needs, we have established Interagency Agreements, memorandums of understanding, and memorandums of need with other agencies. To accomplish our Food Safety Initiative goal (Performance Goal 8 - NARMS), we entered into Interagency Agreements for the development of databases. Therefore, we are dependent on the data validation processes of our sister agencies.

Some of our program work is dependent upon other agencies' planning processes. This is especially true in our illegal residues in meat and poultry program that has responsibility to follow-up on violative tissue residue reported to FDA by USDA's Food Safety and Inspection Service (FSIS). FSIS develops an annual statistical residue sampling plan with input from FDA. However, the majority of violations reported to FDA for investigative follow-up, result from samples from suspect animals. FSIS recently modified sampling criteria, which resulted in an increased number of suspect animals being tested and an increase in violative samples being reported to FDA. Under the new Hazard Analysis Critical Control Point (HACCP) plan, the requirements for how slaughter plants choose samples for testing has also changed substantially so it is extremely hard to judge how many residue reports will be sent to FDA for follow-up investigation.

Medical Devices and Radiological Health

Premarket - To help ensure Agency consistency in tracking and reporting premarket activities, the Medical Device Program utilizes the Premarket Tracking System, which contains various types of data taken directly from the premarket submissions. FDA employs certain conventions for monitoring and reporting performance; among these are groupings of premarket submissions into decision and receipt cohorts. Decision cohorts are groupings of submissions upon which a decision was made within a specified time frame, while receipt cohorts are groupings of submissions that were received within a specified time frame. The premarket performance goals are based on receipt cohorts. Final data for receipt cohorts are usually not available at the end of the submission year. Because the review of an application received on the last day of the submission year, e.g., a PMA with 180 day time frame, may not be completed for at least 6 months or longer, final data for the submission or goal year may not be available for up to a year after the end of the goal year.

Mammography - The Mammography Program Reporting and Information System (MPRIS) is a set of applications used to support all aspects of the FDA implementation of the Mammography Quality Standards Act of 1992. This includes the collection, processing and maintenance of data on mammography facility accreditation and certification, FDA inspections and compliance actions. MPRIS is envisioned as a centralized repository of information that supports FDA's mission to improve the quality of mammography and improves the overall quality, reliability, integrity, and accessibility of facility certification, inspection, and compliance data by eliminating multiple versions of the data while expanding and automating data edits, validation, and security of a single integrated database.

User Facility Adverse Event Reporting - FDA's adverse event reporting system's newest component is the Medical Device Surveillance Network, MedSuN program. MeDSuN is an initiative designed both to educate all health professionals about the critical importance of being aware of, monitoring for, and reporting adverse events, medical errors and other problems to FDA and/or the manufacturer and; to ensure that new safety information is rapidly communicated to the medical community thereby improving patient care.

CDRH Field Data Systems - Data systems include the Program Oriented Data System (PODS) and the Field Accomplishments Tracking System (FACTS). PODS tracks field activities conducted by FDA's field force and the firms over which FDA has legal responsibility. PODS provides most of the information on inspections and other field activities. Field personnel have the major responsibility for assuring the quality of PODS data. CDRH also has its own systems to supplement these Agency systems.

Other Data Sources - These include miscellaneous reports, guides, and files as cited in the data sources for several of the goals.

NCTR

As a research component of the FDA, the National Center for Toxicological Research provides peer-reviewed research that supports the regulatory function of the Agency. To accomplish this mission, it is incumbent upon the Center to solicit feedback from its stakeholders and partners, which include other FDA centers, other government agencies, industry and academia. Scientific program services are provided by the Science Advisory Board (SAB) composed of non-government scientists from industry, academia, and consumer organizations. The SAB is guided by a charter that defines the scope of the review to include quality of the science and the overall applicability to FDA regulatory need. This board is further supplemented with subject matter experts and scientists representing all of the FDA product centers. Programs described are evaluated at least once every five years by the SAB.

Research proposals are monitored through partnerships with other scientific organizations. Scientific and monetary collaborations include inter-agency agreements with other government agencies, Cooperative Research and Development Agreements and technology transfer with industry, and grants or informal agreements with academic institutions.

NCTR uses several strategies to ensure the quality of its research and the accuracy of data collected in specific research studies. Study protocols are developed collaboratively by principal investigators and FDA product centers. Findings are recorded by and verified by internal and external peer review. Statistical analyses are performed by the principal investigator and reviewed by members of the Biometry and Risk Assessment staff. The analytic approach is checked by different members of the scientific staff and the Deputy Director for Research to verify the scientific integrity of the data.

To ensure that the performance data are accurate and timely, the NCTR Planning Division staff monitors research progress at the project level on a recurring basis. The Project Management System utilized by the Planning Staff is capable of tracking planned and actual research projects and expenditures in all three strategic goals and in the outlined performance goals. Quality Assurance Staff monitor the experiments that fall within the Good Laboratory Practices (GLP) guidelines. Research accomplishments and goals are published annually in the NCTR Research Accomplishments and Plans document. Publications reporting research findings are tracked by project, and final reports are archived and distributed to interested parties. Over the past four or five years, NCTR has published yearly 175-250 research documents, manuscripts, book chapters, and abstracts in recognized scientific journals.

NCTR's research findings are also presented at national and international scientific meetings and published in peer-reviewed scientific journals. Many of the scientific meetings are sponsored or co-sponsored by NCTR scientists. The scientists make over 400 presentations and invited speeches a year at local science seminars and at national and international meetings. Many NCTR scientists also serve on international scientific advisory boards.

Previous Page | Table of Contents | Appendix F