TB Notes Newsletter
No. 1, 2012
Evaluation of the National Tuberculosis Indicators Project
Background: In 2006, CDC identified 15 high-priority TB control activities as national TB program objectives. The National Tuberculosis Indicators Project (NTIP) is a secure web-based system established in 2009 to monitor the progress of state and local TB programs toward meeting these objectives. NTIP generates reports for each objective using data already submitted by programs as part of routine TB surveillance. In collaboration with the National Tuberculosis Controllers Association (NTCA), we conducted an evaluation to examine how programs use NTIP and to understand how surveillance data reporting affects NTIP.
Methods: In September 2011, NTCA distributed an online survey about NTIP to all TB controllers and registered NTIP users, representing 50 state and 18 major city or territorial TB programs. We also conducted interviews with CDC NTIP staff and NTIP users about the structure and function of NTIP.
Results: Of 406 people surveyed, there were 122 responses from 38 out of 50 (76%) state and 10 out of 18 (56%) major city or territorial TB programs. The most common job positions of survey respondents were TB program manager (35%), TB epidemiologist (27%), and TB controller (20%). Respondents were most commonly from programs that had 100–500 confirmed TB cases in 2010 (36%) or less than 50 cases (35%).
Responses on the uses of NTIP
The responses indicated that 41% use information from NTIP reports either monthly or quarterly, while 7% use it daily or weekly and 27% use it once or twice a year. A majority of programs (61%) currently use NTIP reports in addition to their own indicators to guide program activities. However, one quarter of respondents said they do not use information from NTIP reports at all, and instead use their own program data.
Programs most commonly use NTIP for preparing interim/annual reports to CDC (58%), summarizing surveillance and/or program data (57%), and checking the completeness and accuracy of data reported to CDC (52%). Of these uses, respondents identified summarizing surveillance and/or program data as the most helpful aspect of NTIP to their current work. The respondents also indicated that their ability to use NTIP effectively would be enhanced by more training on using the NTIP system (43%), more information about NTIP (36%), and further training on the national TB program objectives/performance targets and their rationale (34%).
Responses on surveillance data reporting and NTIP
Approximately one third of respondents stated that the lack of current or accurate surveillance data is the largest barrier to using NTIP effectively. Difficulties transmitting data to CDC have led to discrepancies in the data presented in NTIP reports compared to programs’ own data. Overall, 40% of respondents reported problems uploading surveillance data from their system to CDC since 2009. Only 7% of respondents estimated that the data in NTIP reports matched completely with their own program data. Nearly half of the respondents answered that the data matched 60%–80% of the time, 12% felt the data matched 20%–40% of the time, and 1% stated that nothing matched. One third of respondents reported that NTIP has helped them identify problems with the accuracy of their own surveillance data.
The majority of respondents indicated their program uses a CDC-developed system (the National Electronic Disease Surveillance System [34%] or the electronic Report of Verified Case of Tuberculosis application [27%]) to transmit TB surveillance data to CDC. Other respondents use a commercially purchased system (14%) or a system developed in-house by the TB program (14%). A greater proportion of respondents using commercially purchased or TB program developed systems had problems transmitting data when compared to those using CDC-sponsored systems (Figure 1).
| Figure 1. Percentage of survey respondents reporting TB surveillance data transmission problems by data system type, United States, 2009–2011. |
 |
| Figure 1 shows the percentage of respondents reporting data transmission problems, by data system type. Only 36% of those using CDC-sponsored systems to transmit TB case data reported problems, while 59% of those using in-house systems and 82% of those using commercially purchased systems reported problems. |
Discrepancies can also occur between NTIP reports and local calculations if different data sources or different calculation methods are used. Several respondents pointed to these discrepancies as a reason NTIP reports are not more useful to them locally. In addition to discrepancies between NTIP data and local program data, respondents also pointed out discrepancies between NTIP and National TB Surveillance System data (NTSS). NTIP automatically pulls data from CDC’s dynamic surveillance databases every week. Once the NTSS dataset is “frozen” in the spring of each year, any changes, corrections, or additions to cases counted in previous years will be reflected in NTIP, but will not be updated in NTSS until the following spring. In 2009 and 2010, these discrepancies were complicated when data for three states were manually corrected in NTSS, but not in NTIP. No manual corrections will be necessary in the future, so this source of discrepancies should not recur.
Conclusions: Ensuring that programs are able to transmit TB surveillance data in an accurate and timely manner will allow NTIP to track their progress toward meeting national TB objectives. This information can be especially useful to programs with insufficient epidemiologic support to develop and calculate their own indicators. If significant discrepancies exist between data received at CDC and program data, NTIP will not be able to help programs evaluate their performance.
The following aspects of NTIP can benefit from further examination:
- Timeliness and accuracy of surveillance data transmission
NTIP reports only contain information on TB cases already transmitted to CDC. Programs must be able to submit cases to CDC in a timely manner in order to have up-to-date reports. State and local TB programs can continue to work toward ensuring that they have complete and accurate data on all TB cases in their jurisdiction. Depending on when data are submitted to CDC, programs can then know the best time to generate NTIP reports. CDC can continue to support programs, especially those with in-house or commercially purchased data systems, to ensure correct data transmission.
- Opportunities to assist programs to use NTIP reports
Programs have identified factors that explain why current NTIP methods used to calculate indicators can misrepresent the status of their programs. There must be an understanding as to how indicators are measured at the national level. CDC should expand communication and education about NTIP to ensure programs understand how NTIP reports are generated. Finally, continuing to respond to feedback from local and state TB programs about their experiences with NTIP and the national TB objectives will help guide future improvements.
—Reported by Robert Luo, MD, MPH, EIS Officer,
Kai Young, MPH, and Adam Langer, DVM, MPH
Div of TB Elimination, and
Charles Wallace, PhD, MPH
Texas Dept of State Health Services
Contact Us:
- Centers for Disease Control and Prevention
Division of Tuberculosis Elimination (DTBE)
1600 Clifton Rd., NE
MS E10
Atlanta, GA 30333 - 800-CDC-INFO
(800-232-4636)
TTY: (888) 232-6348 - New Hours of Operation
8am-8pm ET/Monday-Friday
Closed Holidays - cdcinfo@cdc.gov
