Detailed Information on the
Universal Newborn Hearing Screening and Intervention Program Assessment

Identify and implement viable recommendations from the recently completed evaluation of the program.

Increase understanding of the importance of early detection and evidence-based strategies among state agencies, their partners and families to improve follow-up.

Complete an independent evaluation the program.

Measure: Increase the percentage of infants with hearing loss enrolled in early intervention before 6 months of age.

Explanation:Some targets may be shown as NA (not applicable) because program is not proposed for funding.

Measure: Increase the percentage of infants suspected of having hearing loss (based on the results of their newborn hearing screen) who receive a confirmed diagnosis by 3 months of age while maintaining a constant Federal expenditure.

Explanation:Some targets may be shown as NA (not applicable) because program is not proposed for funding.

Measure: Percentage of infants suspected of having a hearing loss with a confirmed diagnosis by 3 months of age.

Explanation:Some targets may be shown as NA (not applicable) because program is not proposed for funding.

Measure: Percentage of Infants with a suspected or confirmed hearing loss referred to an ongoing source of comprehensive health care (i.e. medical home)

Explanation:Some targets may be shown as NA (not applicable) because program is not proposed for funding.

Measure: Increase the percentage of children with non-syndromic hearing loss entering school with developmentally appropriate language skills.

Explanation:Children screened for hearing loss as newborns in the early stage of the program are just now approaching school age. Because service systems were not well developed and data systems were practically non-existent, we estimate a rather small baseline percentage of children with permanent hearing loss nationwide having developmentally approprate language skills at school entry. The target may seem overly ambitious given the baseline. However, given that screening is just about universal in 2005 and both service and data systems have improved and will continue to do so, achieving the target seems realistic. In addition, amplification technology continues to improve rapidly.

Measure: Percentage of infants screened for hearing loss prior to hospital discharge

Explanation:FY 2006 and 2007 targets not applicable because program is not proposed for funding.

Is the program purpose clear?

Explanation: The purpose of the Universal Newborn Hearing Screening and Intervention Program is to ensure that all children who are born with or develop hearing loss early in infancy are identified early and provided appropriate treatment and care that will enable them to develop appropriately for their age.

Evidence: Evidence for this answer can be found in the following documents: 1) Public Law 106-113 2) HRSA direction to grantees: Program application guidance (2005)

Does the program address a specific and existing problem, interest, or need?

Explanation: The program does address a specific and existing problem as congenital deafness remains the most common of birth defects, affecting 1 to 3 infants per thousand. Prior to the development of technology (in the late 1980s) that made it possible to screen a newborn infant for hearing loss, this diagnosis was typically not made until the child was about 2-3 years of age. Late identification of hearing loss leads to the need to remediate language/speech in special education in a population whose only disability is their inability to hear. Despite special education services, children who are not exposed to spoken language early in life read, on average, at a 4th grade level at high school graduation. These statistics highlight the great need for a program that works to ensure that all infants have their hearing tested at birth (and several months after) and that those with hearing loss are entered into early intervention programs that enable them to meet developmental milestones in language and cognition.

Evidence: Evidence for this answer can be found in the following documents: 1) White, Karl R., The evolution of early hearing detection and intervention programs in the United States. 2) Harrison and Rousch in Spivak, L. Universal newborn hearing screening, Thieme 1998 p5. 3) Keren, et al (2002): Projected Cost-effectiveness of statewide newborn hearing screening. Pediatrics 110:5, 855-864.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program is not redundant or duplicative of any other Federal, state, local or private effort, as its authorizing legislation outlined a specific role for the program. The legislation calls for the program to provide support- including funding, program guidance and technical assistance- to States so that they can develop the clinical services of screening, primary care, diagnosis and referral to early intervention. No other agency has this role.

Evidence: Legislation does call for the program to collaborate with other agencies on this issue: CDC - to develop relevant data and tracking systems. NIH/NIDCD - to conduct research on technology and language.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program is free of design flaws that prevent it from meeting its defined objectives and performance goals and there exists no strong evidence that another approach would be more efficient or effective in achieving the program's intended purpose.

Evidence: The following two studies provide evidence of the program's cost effectiveness: 1) ""Projected Cost-Effectiveness of Statewide Universal Newborn Hearing Screening"", Keren et al. 2) ""Cost and cost-effectiveness of universal screening for hearing loss in newborns"", Kezirian, et. al., In addition, there is no evidence that an alternative design for this program would be more efficient.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program is effectively targeted as its resources reach its intended beneficiaries and directly address the program's purpose. Funds are provided directly to the state entity responsible for the development of the infrastructure that enables the State to implement statewide newborn hearing screeing programs. These programs not only ensure that infants are screened for hearing loss prior to their discharge, but that an audiologic diagnosis is made before 3 months of age for those infants who do not pass the screening test, and that infants with confirmed hearing loss are enrolled in early intervention by 6 months of age.

Evidence: Evidence for this answer can be found in the following document and website: 1) Program Guidance for New and Continuing Discreationary Grants (2005) 2) www.infanthearing.org

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has two new long-term measures, one of which has a baseline at target, and the other for which a baseline and target is currently under development. One of these will directly measure health outcomes of the programs intended beneficiaries and the other measures this indirectly. Both measures were designed to meaningfully reflect the purpose of the program.

Evidence: The program's long-term measures are as follows: 1) By 2012 increase the percentage of infants with hearing loss enrolled in early intervention before 6 months of age. 2) By 2012 increase the percentage of children with non-syndromic hearing loss entering school with developmentally appropriate language skills.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program does have an ambitious target and timeframe for one of its long-term measures: By 2012 increase the percentage of infants with hearing loss enrolled in early intervention before 6 months of age from 57% in 2004 to 85% in 2012. The program is currently developing a baseline and target for its second long-term measure that will assess the program's impact on the health and/or development of children identified early in life with hearing loss. The baseline and targets for this measure will be made available in the Fall (by Nov. 1st, 2005) for inclusion in both the PART document and the FY 2007 Budget.

Evidence: The following individuals/experts can attend to the ambitiousness of the program's long-term goal: 1) Dr. Karl White, Director of the MCHB funded national technical resource center, and internationally recognized expert in the development of newborn hearing screening programs, 2) Dr. Coleen Boyle, Director of Policy for the National Center on Birth Defects and Developmental Disabilities (NCBDDD) at the CDC 3) Jodi Chappell, Director of Policy, at the American Academy of Audiology (AAA) 4) Pam Mason, Director of Audiological Practices Unit at the American Speech Hearing and Language Association (ASHA) .

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program does have several specific annual measures that can demonstrate the program's progress toward achieving its long-term goals.

Evidence: The program's annual measures are as follows: 1) Percentage of infants screened prior to discharge. 2) Percentage of infants suspected of having hearing loss with a confirmed diagnosis by 3 months of age. 3) Percentage of infants with a suspected or confirmed hearing loss referred to an ongoing source of comprehensive health care (i.e. medical home)

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program's baselines and targets for each of its annual measures are ambitious: 1) Increase the percentage of infants suspected of having a hearing loss with a confirmed diagnosis by 3 months of age from 50% in 2004 to 70% in 2007. 2) Increase the percentage of infants with a suspected or confirmed hearing loss referred to an ongoing source of comprehensive health care from 77% in 2004 to 90% in 2007.

Evidence: Evidence for this answer can be found in the follow report: "The evolution of Early Hearing Detection and Intervention Programs in the United States" by Dr. Karl White, Director of the Maternal and Child Health Bureau's national technical resource center, and internationally recognized expert in the development of newborn hearing screening programs.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The program has demonstrated a high level of coordination with and collaboration among Federal and State agencies, as well as professional organizations and advocacy groups. The program's guidance (2005) requires evidence of collaboration between the newborn hearing screening program and the State Title V agency. In most States, the Universal Newborn Hearing Screening program is either within or co-located with the Title V program.

Evidence: The program's partners include other Title V grantees (Maternal and Child Health Block Grant programs), CDC, NIH/NIDCD, the Department of Education, the American Academy of Pediatrics and Family Voices. For evidence of the program's coordination with these various groups/organizations/agencies one may refer to the following documents: 1) minutes of advisory committees 2) AAP Chapter Champions work plans 3) co-sponsored grantee meetings 4) intra-agency agreements

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program recently contracted with Mathematica Policy Research, a well recognized leader in evaluation studies, to conduct an independent evaluation of the MCHB newborn hearing screening program. The scope and quality of the evaluation are defined by the survey tool which covers all aspects of the newborn hearing screening and intervention program, and is to be used with all states. The tool is currently in the OMB clearance process.

Evidence: The tool is currently in the OMB clearance process and was provided in the program's evidence book. The Statement of Work for that contract is also available.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: This question is deemed "Not Applicable" as no funding was requested for this program in the FY 2006 Budget.

Evidence:  

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: There are no strategic planning deficiencies for the program to correct.

Evidence:  

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program's grantees are required to report annually on progress toward program goals. Data is also gathered from State Title V Programs, the National Center on Hearing Assessment and Management, NIH, and the Department of Education and CDC (the CDC collects information on newborn hearing screening and followup). In 2005, grantees for this program will begin to address designated performance measures both in the application and reporting process through: 1) the HRSA Electronic Handbook and 2) MCHB Discretionary Grant Information System (DGIS).

Evidence: Evidence for this answer can be found in the following documents and/or web-sites: ?? Discretionary Grant Performance Measure for Early and Continuous Screening ?? Title V Block Grant Performance Measure for Newborn Hearing Screening ?? GPRA Performance Measure for Newborn Hearing Screening ?? Healthy People 2010 Measure for Newborn Hearing Screening ?? NCHAM website available at www.infanthearing.org Additional evidence for this answer can be found at the CDC web site: www.cdc.gov/ncbddd/ehdi

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers of the program are held accountable for the performance of the program through the HRSA Performance Appraisal Program's Performance Evaluation Plan. This evaluation specifically inquires as to whether or not the manager's program is achieving its desired results. Grantees are held accountable by the requirement (discussed above) that they submit data to the electronic Discretionary Grant Information System (DGIS).

Evidence: Accountability for cost, schedule, and performance results are maintained through: 1) the Annual Operating Plan 2) HRSA's Functional Grants Process Flow Chart 3) HRSA's Procurement Plan and Timelines 4) HRSA's Office of Performance Review

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The Universal Newborn Hearing Screening (UNHS) program has several tracking mechanisms that are used by the program, MCH Bureau management and HRSA to assure that scheduled deadlines for each step in the grants management process are met. The program has always met its deadlines and provided funds to grantees on schedule. One major tracking mechanism is the HRSA Grants Matrix which was established as a project management tool by which to track the efficiency of grant program administration. The UNHS program adheres to the milestone dates published in the Grants Matrix throughout the annual grants cycle to ensure that funds are obligated in a timely manner. In addition, the Newborn Hearing Screening program has not once lapsed funds during its 5 year lifetime.

Evidence: In addition to the HRSA Grant Matrix, the program is held accountable for obligating their funds in a timely manner through the following mechanisms: 1) The program's Annual Operating Plan, which forecasts and monitors program expenditures throughout the fiscal year. 2) The HRSA Functional Grants Process Flow Chart 3) The HRSA Procurement Plan and Timelines 4) The Maternal and Child Health Bureau Status of Funds Monthly Report

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has developed a new measure with a baseline and targets that will enable it to meaure annually the program's success in achieving greater efficiencies.

Evidence: The program's efficiency measure is as follows: Increase the percentage of infants, suspected of having hearing loss (secondary to the results of their newborn hearing screen), who receive a confirmed diagnosis by 3 months of age while maintaining a constant Federal expenditure.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program both coordinates and collaborates effectively with the CDC and the NIH to achieve its primary goals and objectives. 39 State programs receive funds from both HRSA and the CDC to address the respective purpose of: 1) HRSA - screening and providing access to services 2) CDC - surveillance and tracking The program collaborates with NIH/NIDCD to develop a research agenda that looks at language development in young children with hearing loss (currently six studies are in progress), the efficacy of screening technologies, and the genetics of hearing loss.

Evidence: The program collaborates and coordinates with CDC and NIH through: 1) a HRSA sponsored Interagency Coordinating Committee (minutes of a meeting provided as evidence). 2) Jointly sponsored grantee meetings 3) Interagency agreements (with CDC) 4) Joint site visits and presentations. 5) Jointly planning and funding the CDC FY 2002 One Percent Evaluation Proposal on Assessment of Loss to Follow-up in State Early Hearing Detection, and Intervention (EHDI) Programs.

Does the program use strong financial management practices?

Explanation: In FY 2004, HHS OIG conducted an HHS financial statement audit. The audit reported that the Department had serious internal control weaknesses in its financial systems and processes for producing financial statements. OIG considered this weakness to be material. The audit recommended that HHS improve their reconciliations, financial analysis, and other key controls. The September 30, 2002 HRSA independent auditor's report found that the preparation and analysis of financial statements was manually intensive and consumed resources that could be spent on analysis and research of unusual accounting. The audit also found that HRSA's interagency grant funding agreement transactions were recorded manually and were inconsistent with other agencies' procedures. Finally, the audit found that HRSA had not developed a disaster recovery and security plan for its data centers.

Evidence: Evidence for this answer can be found in the following documents: 1. HHS FY 2004 Performance and Accountability Report 2. HRSA's 2002 audit report

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The DHHS' long-term strategic plan to resolve the internal control weaknesses is to replace existing accounting systems and other financial systems within HHS with the Unified Financial Management System (UFMS). HHS plans to fully implement the UFMS Department-wide by 2007. HRSA developed a corrective action plan to address the reportable conditions identified in the September 30, 2002 independent auditor's report. For each aspect of the five reportable conditions, HRSA assigned an office responsibility. The plan also outlines milestones and target completion dates. When the new performance measures are in use, results from these measures will be available to public via the HRSA website.

Evidence: Evidence for this answer can be found in the following documents: 1. HHS FY 2004 Performance and Accountability Report www.hhs.gov/of/reports/account/acct04/pdf/section4.pdf 2. HRSA Corrective Action Plan for FY 2002 Financial Statement Audits

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: 100 percent of the grants awarded by the Universal Newborn Hearing Screening program are reviewed, ranked, and awarded according to a competitive process established by the centralized HRSA Division of Independent Review (DIR). There are no earmarks and all awards are subject to peer review.

Evidence: Evidence for this answer can be found in the following documents: ?? Excerpt from HRSA Preview regarding eligibility ?? Ranking document for HRSA Newborn Hearing Screening Program ?? Description of the DIR available at the www.hrsa.gov

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The program has a reporting system in place that enables it to document grantees' use of funds in eligible activity categories. Each grantee submits an annual progress report and continuation application which is reviewed an monitored by both the program staff and the grants management officer. In addition, program managers monitor the expenditure of funds through-out the year through the MCHB Monthly Status of Funds Report.

Evidence: In 2005 the programs ability to oversee grantees' activities will be greatly enhanced with the implementation of the electronic application and reporting system established by the Electronic Handbook (EHB) and the Discretionary Grant Information System (DGIS).

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Performance data (including number of infants screened, number diagnosed by 3 months of age and enrolled in early intervention by 6 months) are collected annually by the program, the national technical resource center, NCHAM, and the CDC.

Evidence: Evidence for this answer can be found on the MCH data web site (www.mchdata.org), the National Center on Hearing Assessment and Management web site, and www.infanthearing.org.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program's primary historical long-term measure, the percentage of newborn infants screened for hearing loss prior hospital discharge, has demonstrated that the Universal Newborn Hearing Screening Program has made great progress in achieving its primary goal. In 1999, only 34% of newborns were screened for hearing loss prior to discharge, by 2004 this percentage had increased to 95%. Because the program has largely achieved its initial goal, it has developed two new long-term measures (as indicated above in the answer to question 2.1) that enable the program to now measure its effectiveness in improving the long-term health outcomes of children found, through newborn testing, to have hearing loss. The baseline and targets for one of these new measures is currently under development, but will be made available by the program this Fall for inclusion in both this PART and the FY 2007 Budget.

Evidence: UNHS GPRA measures - FY 2006 Congressional Justification

Does the program (including program partners) achieve its annual performance goals?

Explanation: Of the program's three annual measures, two are newly developed and have no historical data that can be used to inform the program's progress towards achieving those performance goals. Looking at the performance measure for which there is historical data (percentage of infants screened prior to hospital discharge) the program reached (and surpassed) its target every year between 1999 and 2004.

Evidence: UNHS GPRA measures - FY 2006 Congressional Justification

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has demonstrated improved efficiencies and cost effectiveness in achieving its goals each year. Beginning in FY 2002 the program has experienced minor decreases in its budget yearly from $9.995 million (in FY 2002) to $9.935 million in FY 03, $9.873 million in FY 04 to $9.792 million in FY 05. Despite these minor yearly decreases the program demonstrated yearly increases (FY 2002- 85%, FY 2003- 89.3%, FY 2004- 95%) in the percentage of newborns screened for hearing loss prior to hospital discharge.

Evidence: The program's efficiency measure, that demonstrated the efficiency described in the explanation to this answer, is as follows: Increase the percentage of infants screened for hearing loss prior to hospital discharge while maintaining a constant (or in this case slightly decreasing) Federal expenditure. (Please note: As explained in the answer and evidence for question 3.4, the program has developed a new measure that will allow it to measure its efficiency starting in FY 2005.)

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There is no other Federal, State, local, private or non-profit organization that is currently engaged in activities similar to those performed by the Universal Newborn Hearing Screening program. No other public program or non-government organization has committed itself to ensuring that all children who are born with or develop hearing loss are identified early and provided appropriate treatment and care that will enable them to develop appropriately for their age.

Evidence: In addition, universal newborn hearing screening did not exist, in any venue, public or private, before the development of this program. It continues to be the only program that performs these functions.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: While the program has indicated that an independent evaluation of the program is in the planning stages, there has been no evaluation of sufficient quality and scope performed in the past that can inform "yes" or "small extent" answer to this question.

Evidence: