Detailed Information on the
Abstinence Education Assessment

Establishing baselines for the program-specific annual measures and the efficiency measure.

Implementing changes in data collection process to gather program-specific data on rates of abstinence pre- and post-program participation, number of youths served, and hours of intervention provided.

Initiating a rigorous, experimental evaluation of the community-based discretionary grant program. The program will also establish a plan to disseminate research and best practices findings among grantees.

Establishing baselines for the program-specific annual measures and the efficiency measure. Milestone: Convene a meeting of abstinence education researchers and practitioners to inform development of program specific performance measure.

Initiating a rigorous, experimental evaluation of the community-based discretionary grant program. It will also establish a plan to disseminate research and best practices findings among grantees. Milestone: Office of the Assistant Secretary for Planning and Evaluation (ASPE) (under intra-agency agreement with Family and Youth Services Bureau) to award contract to Abt Associates to survey programs with various approaches to serving high school students.

Implementing changes in data collection process to gather program-specific data on rates of abstinence pre and post program participation, number of youths served, and hours of intervention provided. Milestone: In collaboration with experts, program will develop data collection forms.

Measure: Decrease the rate of births to unmarried females (i.e., number of births per 1,000 females), aged 15-19, by 1.0 births from the FY 2008 actual, per 1,000 unmarried teenage girls as measured by NVSS by FY 2014.

Explanation:

Measure: Cost of program delivery per youth.

Explanation:(FY 200X program budget) / (FY 200X # of youth served) = FY 200X cost per youth served 1.) Not every state will have reported the number of youth served by August 2006. Thus, the numerator (the program budget) will be reduced by the amount of grant monies given to states that have not reported the number of youth served. The number will be adjusted later when all states that accepted grants have reported the number of youth served. 2.) At present, this efficiency measure only focuses on Title V (i.e. mandatory) grant funding and youth served. The Community-Based Abstinence Education (CBAE) program (i.e. the discretionary program) grantees have recently been asked to report the number of youth served; thus, in future years, this measure will include CBAE funding and youth served.

Measure: Increase the proportion of youth who have never had sexual intercourse and remain abstinent following participation in an abstinence education program.

Explanation:

Measure: Increase the proportion of youth who have had sexual intercourse but have discontinued having sex following participation in the abstinence education program.

Explanation:

Measure: Decrease the proportion of students in grades 9-12 who have ever had sexual intercourse to one percentage point below the FY 2009 actual result by FY 2014 as measured by YRBSS.

Explanation:For both the proportion of students in grades 9-12 who have ever had sexual intercourse and the proportion of students in twelfth grade who have ever had sex, the Abstinence Education programs are aiming for bi-annual decreases of 0.5 percent. The historical data, most recent data (baseline) and targets are shown in the tables below.

Is the program purpose clear?

Explanation: The mandatory Title V, Section 510 Abstinence Education Program provides formula grants to States for abstinence education. States, in turn, use the grants to promote abstinence from sexual activity until marriage among a targeted group of youths who are most likely to bear children out-of-wedlock. The discretionary Community-Based Abstinence Education (CBAE) Program awards grants to public and private entities for implementation of abstinence education programs for adolescents ages 12 through 18. CBAE grantees focus on educating young people and creating an environment within communities that supports teen decisions to postpone sexual activity until marriage. Examples of the Title V and CBAE abstinence education program activities include mentoring, counseling, and teaching abstinence-only education curriculum to youths in public schools and other community settings. Both programs also adhere to the following definition of abstinence education as defined in Title V, Section 510 of the Social Security Act. (A) has as its exclusive purpose, teaching the social, psychological, and health gains to be realized by abstaining from sexual activity; (B) teaches abstinence from sexual activity outside marriage as the expected standard for all school age children; (C) teaches that abstinence from sexual activity is the only certain way to avoid out-of-wedlock pregnancy, sexually transmitted diseases, and other associated health problems; (D) teaches that a mutually faithful monogamous relationship in context of marriage is the expected standard of human sexual activity; (E) teaches that sexual activity outside of the context of marriage is likely to have harmful psychological and physical effects (F) teaches that bearing children out-of-wedlock is likely to have harmful consequences for the child, the child's parents, and society; (G) teaches young people how to reject sexual advances and how alcohol and drug use increases vulnerability to sexual advances; and (H) teaches the importance of attaining self-sufficiency before engaging in sexual activity.

Evidence: 1) Social Security Act § 510 (42 U.S.C. 710) ("Title V"); 2) Social Security Act § 1110 (42 U.S.C. 1310) ("Title XI"); 3) Appropriation Language, Excerpts from the Departments of Labor, Health, and Human Services, and Education, and Related Agencies Appropriations Act, 2006 - Public Law 109-149; 4) House Committee Report, Excerpts from the Departments of Labor, Health, and Human Services, and Education, and Related Agencies Appropriation Bill, 2006 (Report No. 109-143); and 5) Senate Committee Report, Excerpts from the Departments of Labor, Health, and Human Services, and Education, and Related Agencies Appropriation Bill, 2006 (Report No. 109-103).

Does the program address a specific and existing problem, interest, or need?

Explanation: The Abstinence Education program addresses an existing problem: the high rates of teen sex and the resulting consequences that affect millions of lives. The 2003 CDC's Youth Risk Behavior Surveillance System (YRBSS) show that 46.7 percent of high school students in 9th through 12th grade report having had sexual intercourse, and this number rises to 61.6 percent for high school seniors. Concurrently, teenagers are disproportionately at risk of acquiring sexually transmitted diseases (STD's). According to CDC reports, adolescents make up 20% of the U.S. population but one-fourth of the approximately 15 million new cases of sexually transmitted disease acquired by Americans each year are among adolescents. Additionally, the younger an adolescent is when he or she initiates sexual activity, the more sexual partners he or she is likely to encounter and, therefore, the greater the risk of acquiring a sexually transmitted disease. Teen sexual activity also leads to unintended teen pregnancies: 34 percent of women become pregnant at least once before they reach the age of 20. Eight in ten of these pregnancies are unintended. In fact, the teen pregnancy and birth rates for the United States are higher than any other fully developed nation. For example, the U.S. pregnancy rates are nearly twice as high as rates in Canada and England and seven to eight times as high as rates in Japan and the Netherlands. Finally, teens' future prospects decline when teens give birth. Teen mothers are less likely to complete high school, more likely to be single parents, and more likely to live in poverty than other teens. The children of teenage mothers do not fare better. They have lower birth weights, are more likely to perform poorly in school, and are at greater risk of abuse and neglect.

Evidence: 1) The Youth Risk Behavior Surveillance System (YRBSS); 2) Annie E. Casey Foundation. (1999). When Teens Have Sex: Issues and Trends, a KIDS COUNT Special Report. Washington, DC.; and 3) National Campaign to Prevent Teen Pregnancy (2003). With One Voice 2003: America's Adults and Teens Sound Off About Teen Pregnancy. Washington, DC.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Title V and CBAE abstinence education program are the only Federal program, which focuses on promoting abstinence until marriage. Other federal programs, such as Title X (family planning), the HHS Adolescent Family Life Program (AFL), and USAID Abstinence and Healthy Choices for Youth each have a purpose that is distinct from the abstinence education program and thus fund a different set of activities. Specifically, the differences are as follows: 1) Title X (family planning) funds support educational strategies for safer sex, such as correct usage of contraception and condoms; 2) The legislation authorizing the Adolescent Family Life program allows for abstinence only education through its "prevention" grants; however, it provides services beyond abstinence education for its "care" grants such as pregnancy testing and maternity counseling, nutrition information and counseling, and appropriate educational and vocational services; and 3) The USAID's Abstinence and Healthy Choices for Youth program targets a broader population of youths ages 10 through 24. It also supports prevention programs that follow the ABC guidelines, which stands for "abstinence, be faithful, and correct and consistent use of condoms." Also, Title V and CBAE programs are not redundant of other non-governmental programs. Perhaps the greatest efforts outside of government are the work of houses of worship. Differences with the Federal abstinence until marriage programs include using sacred texts as a basis for teachings related to sexuality, leaders or curriculum that promote contraception, and the extent to which abstinence until marriage is an explicit goal. For example, one study of leaders from Protestant (38 different denominations), Roman Catholic, Unitarian Universalist, Jewish and Islamic traditions found that of their survey participants, 62 percent felt that faith-based institutions should teach teenagers both abstinence and comprehensive sex education, 30 percent preferred an abstinence-only approach, 4 percent felt that sex education should only be in the home, and 4 percent favored comprehensive sex education without an abstinence emphasis.

Evidence: 1) Clapp, S., Leverton Helbert, K., & Zizak, A. (2005). Faith matters: Teenagers, sexuality and religion. Fort Wayne, IN: Religious Institute on Sexual Morality, Justice and Healing; 2) Religious Institute on Sexual Morality, Justice and Healing. (2003). Youth development and faith-based institutions: Religious Institute study notes. Fort Wayne, IN; 3) ABC Guidance #1 for United States Government In-Country Staff and Implementing Partners Apply the ABC Approach to Preventing Sexually-Transmitted HIV Infections Within the President's Emergency Plan for AIDS Relief, The President's Emergency Plan for AIDS Relief, Office of the U.S. Global AIDS Coordinator, Final; 4) Public Health Service Act § 1001 ("Title X - Population Research and Voluntary Family Planning Programs"); 5) Public Health Service Act § 2001 ("Title XX - Adolescent Family Life Demonstration Projects); 6) Social Security Act § 510 (42 U.S.C. 710) ("Title V"); and 7) Senate Committee Report, Excerpts from the Departments of Labor, Health, and Human Services, and Education, and Related Agencies Appropriation Bill, 2006 (Report No. 109-103).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: To examine potential flaws in the program design, it is helpful to assess how well the current design allows the program to meet its mission. Title V and CBAE aim to educate youths to abstain from sexual activity until marriage and thus, avoid negative health and social consequences. There are other programs that teach youths about safer sex such as use of contraception; however, Title V and CBAE abstinence education are the only program that teache youths to abstain from sexual activity until marriage. In other words, the program design focuses on abstinence, which is the only certain way to avoid pregnancies and sexually transmitted diseases. According to data from the National Longitudinal Study of Adolescent Health, young women who take a virginity pledge are 40 percent less likely to have a child out-of-wedlock than similar young women who do not make pledges. These findings are valid even when factors such as socio-economic status, race, religion, and other relevant variables are held constant. Also, there are unique characteristics of the program's formula and competitive grant designs that maximize the program's efficiency. The Title V formula grants to States allow for a nationwide distribution of abstinence education funding and coordination of service delivery across the State in ways that local grants cannot. On the other hand, the discretionary competitive grants target communities that demonstrate a great need such as higher than average rates of adolescent births. The discretionary grants encourage innovation, and provide services that are tailored to the unique needs of the target local population. Another key aspect of the program design is that grantees must spend 15% of grant awards on evaluation. The grants are intended to spur innovation and flexibility; therefore, a wide range of approaches to abstinence education is encouraged. The evaluation requirement ensures that all approaches are evaluated and that ineffective approaches are not funded again.

Evidence: 1) ACYF/FYSB; FY 05 Community-Based Abstinence Education Program Announcement, HHS-2006-ACF-ACYF-AE-0099, CFDA#93.010; 2) ACYF/ FYSB; FY 06 Section 510 Abstinence Education Program Announcement, ACYF-FYSB-AE-01-06, CFDA#93.235; and 3) Kirk A. Johnson and Robert Rector, "Adolescents Who Take Virginity Pledges Have Lower Rates of Out-of-Wedlock Births," The Heritage Foundation, forthcoming, April 2004.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The statutory language for the Title V and CBAE programs require that grants are awarded to provide abstinence education, which target youths at risk for out-of-wedlock births and youths ages 12 through 18. The program ensures that grantees adhere to the statutory mandate by requiring applicants to identify the target population and age range, as well as demonstrate the need for abstinence education among the proposed target population. Many factors have been associated with an increased risk of pregnancy among American youths. Examples include youths belonging to an ethnicity with higher than average rates of teen pregnancies, living in foster care, and growing up in a low-income family. Grant reviewers and Federal staff consider such factors when reviewing applications for how well the grant proposals reflect a demonstrated need for abstinence education among the proposed target population. The formula based Title V grants to States take an added measure to assure that grants reach those groups most likely to bear children out of wedlock. Grant awards are appropriated based on the proportion of low-income children in the State compared to the total number of low-income children in all States, according to the latest Census data. The formula responds to the research that shows the rate of births among poor teens is three times higher than that of non-poor teens. For discretionary CBAE abstinence program, consideration is given to geographical distribution of grants to ensure wide distribution of funding. For example, the program may fund an applicant from a rural county who may score lower in the grant review process rather than fund multiple applicants from an urban county. Finally, there are standard grant management procedures that staff follow to ensure that resources reach intended beneficiaries for both the Title V and CBAE programs. Routine monitoring includes phone calls, site visits and review of semi-annual reports for both programmatic and fiscal issues. Further, technical assistance is provided to enhance grantees' impact by tailoring the grantees' individual abstinence education programs to meet cultural and other special needs of high-risk target populations.

Evidence: 1) Social Security Act § 510 (42 U.S. C. 702) ("Section 502"); 2) Social Security Act § 510 (42 U.S.C. 710) ("Title V"); 3) ACYF/FYSB; FY 05 Community-Based Abstinence Education Program Announcement, HHS-2006-ACF-ACYF-AE-0099, CFDA#93.010; and 4) ACYF/ FYSB; FY 06 Section 510 Abstinence Education Program Announcement, ACYF-FYSB-AE-01-06, CFDA#93.235.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has performance measures, which capture national rates of sexual intercourse among high school students and rates of out of wedlock births among adolescents. This is an appropriate measure because the program funding has a wide reach as evidenced by the Title V mandatory funding allocated to States by formula, and the discretionary funding for communities across the U.S.

Evidence: See measures tab.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has baselines and ambitious targets for the national trend measures. The targets continue to push for lower out-of-wedlock births and lower rates of sexual activity.

Evidence: See measures tab.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has newly established performance measures, which capture the proportion of youths who abstain from sex upon participation in the abstinence education program. Data trends for the annual measure, when analyzed together with the long-term measure, will show the program specific impact as well as its contribution to the national trend.

Evidence: See measures tab.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: While the program has ambitious targets forthe proposed annual measures, it lacks baselines. The program will establish baselines for FY 2008 in March 2009.

Evidence: See measures tab.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: While partners support the overall goal of the program, they do not currently measure and report on their performance as it relates to accomplishing the program's long-term and annual goals. The program is taking steps to develop appropriate performance measures and has taken steps to begin collecting output data, including number of youths served, hours of service provided, and number of youths graduated from the program.

Evidence: Training Model for Abstinence Education Grantees: Measuring Program Results.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: An independent evaluation group is conducting a multi-year evaluation of the Title V abstinence education program. The study consists of an implementation and process analysis, as well as an impact evaluation. The former is intended to help the program learn the most effective ways for implementing the program where as the latter will help the program learn about the specific mechanisms through which the program can cause behavioral changes. These findings will help the program in continually improving the program design and guide the program towards the elements that it should look for when awarding grants. The impact evaluation uses a rigorous experimental design with random assignment of control and experimental group. Also, the program is working with HHS' planning and evaluation office to evaluate the discretionary CBAE abstinence education program. While this is in the planning stages, the study will follow adolescents through high school and will measure the impact of the program on behavioral outcomes, including rates of out-of-wedlock pregnancies and sexually transmitted diseases.

Evidence: Maynard, R. A., Trenholm, C., Devaney, B., Johnson, A., Clark, M. A., Homrighausen, J., & Kalay, E. (2005). First-Year Impacts of Four Title V, Section 510 Abstinence Education Programs. Princeton, NJ: Mathematica Policy Research Inc;

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: While the program provides an estimate of the number of new grants it can award with additional funding, it does not explain how the new grants will impact the annual and long-term goals. In other words, it is unable to clearly define a direct relationship between the performance goals and resources. However, the program presents resource needs in a complete and transparent manner, as evidenced by the "Resource and Program Data Sheet." The data sheet shows funding designated for grants, research and evaluation, demonstration and development, training and technical assistance, and program support. Included in program support are indirect costs for information technology support, grant/ paneling review, contract fees, printing, and associated overhead costs.

Evidence: Congressional Justification, Department of Health and Human Services, Administration for Children and Families, Fiscal Year 2007, Justification of Estimates for Appropriations Committees.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program is strengthening its strategic planning by working to identify and correct existing weaknesses. The program has measures in place, which assess national rates of sexual activity and out of wedlock births. However, it recently noted the challenges of linking the program's direct impact on the national rates. The program is working to address this problem, as evidenced by the developmental performance measure which gets closer to the program's direct impact. This proxy measure, in concert with on-going program evaluations will help the program identify outcomes of the abstinence education program.

Evidence: Since the abstinence education program was recently transferred from the Health Resources and Services Administration (HRSA), the focus was on program implementation. For example, the program focused on refining the program design to be consistent with a social services approach rather than a public health approach. Now that grants have been awarded successfully in the past few years, the program is directing its attention to strategic planning matters. In June of 2006, the program noted the weaknesses in current performance measures. As a result, they began exploring program specific performance measures. The abstinence education program considered measuring abstinence rates of program participants; however, it concluded that this was not a viable option because of the difficulty of locating and following up on program participants after they graduate from the abstinence education program. The program continues to explore appropriate program specific performance measures to assess its impact.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program does not regularly collect high-quality performance data related to program goals and has not established baseline data to set meaningful ambitious performance targets. However, it recently took steps to collect data on the number of youths served, hours of service, and number of youths who graduate from the abstinence education program. It also collects information on grantees' major activities and accomplishments, whether grantees experienced problems and how they were addressed, amount of funds spent compared to the award amount, program costs, and overhead costs compared to service costs. While this is helpful information, they do not reflect high quality performance data that is collected through a systematic process with quality controls to confirm the validity of the data.

Evidence: 1) ACYF/FYSB; FY 05 Community-Based Abstinence Education Program Announcement, HHS-2006-ACF-ACYF-AE-0099, CFDA#93.010; 2) ACYF/ FYSB; FY 06 Section 510 Abstinence Education Program Announcement, ACYF-FYSB-AE-01-06, CFDA#93.235; 3) Community-Based Abstinence Education Grant Program Quarterly Grant Status Update; 4) Community-Based Abstinence Education Grant Program Site Visit Monitoring; and 5) Standard Form 269 (Rev.7-97) Financial Status Report (Long Form)

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers have clearly defined performance standards. The Assistant Secretary for Administration for Children and Families (ACF) has a performance plan that holds him responsible for improving financial management, expanding electronic government and improving budget and performance integration that apply to all ACF program including Abstinence Education. He is also responsible for "developing a baseline for the number of youth who receive abstinence until marriage education." The Associate Commissioner for the Families and Youth Services Bureau (FYSB) has a performance plan that includes ensuring financial and managerial accountability and achieving key program and management outcomes. Similar to the Assistant Secretary of ACF, the Associate Commissioner is charged with developing a baseline for the Title V and CBAE abstinence education programs. Other performance objectives include developing data collection standards and increasing the program's reach in the number of youths served. This accountability filters down to the program staff whose performance plans include clearly defined performance objectives, and standards for program implementation. Grantees are required to submit monthly or quarterly projections of expected outcomes in the initial grant application. This may include such items as the number of youths served, hours of services provided, and new activities or services that will be implemented. Grantees are held accountable to the targets identified in the application. They are also held accountable through reporting, monitoring, and the standard terms and conditions of the grant. Grants awarded are multi-year projects and grantees are advised that continuation of funding is dependent on satisfactory progress. Further, grantees are informed that failure to submit reports on time may be the basis for withholding financial assistance payments, suspension, termination or denial of refunding.

Evidence: 1) Performance Plan for the Assistant Secretary of Administration for Children and Families; 2) Performance Plan for the Associate Commissioner of Administration for Children and Families; 3) Performance Plan for Supervisory Abstinence Program Specialist; 4) Standard Terms and Conditions for recipients of HHS/ACF Financial Assistance Award; and 5) Standard Form 269 (Rev.7-97) Financial Status Report (Long Form)

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: While the discretionary CBAE abstinence education program does not have the problem of unobligated funds, the mandatory Title V block grant has had this problem since the passage of the legislation that established the program. For example, about $9 million remained unobligated from the Title V program in FY 2005. This is because there were States or territories that chose to not accept the Federal abstinence education funding. However, the program has adequate procedures to formulate spending plans and report actual expenditures. For example, the program, in coordination with the ACF bureau's budget office, completes a "Resource and Data Sheet" to track how funds are allocated and actual expenditures. The program also uses the Grants Administration Tracking and Evaluation System (GATES) to make prompt and accurate grant awards. The program also uses GATES and financial status report (SF-269A) to ensure that grantees obligate funds in a timely manner and spend funds for intended purposes. Title V grantees must submit SF-269A reports annually and the CBAE grantees must submit them semi-annually.

Evidence: 1) Congressional Justification, Department of Health and Human Services, Administration for Children and Families, Fiscal Year 2007, Justification of Estimates for Appropriations Committees; 2) Standard Form 269 (Rev.7-97) Financial Status Report (Long Form)ACF Grantee Financial; and 3) Grants Administration Tracking and Evaluation System (GATES). Access at: https://extranet.acf.hhs.gov/ssi/ (passwords are needed for access).

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has newly developed efficiency measure, but has yet to identify baselines and set targets. Also, there are no notable procedures for increasing efficiency and cost effectiveness outside of the grant design.

Evidence: The program proposes to track efficiency by measuring the cost of program delivery per youth.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The abstinence education program has meaningful collaborations and partnerships. For example, it has a memorandum of understanding and intra-agency agreement with the Office of Population Affairs (OPA) at HHS to provide a National Abstinence Education Media Campaign. This campaign consists of the development and production of parent and teen guide publications, a website, and other media messages (billboards, radio, transportation, television and print). The program also partnered with OPA to sponsor an abstinence evaluation conference by entering into an intra-agency agreement for $265,000. The conference accommodated the maximum capacity of more than 300 participants. Finally, the program has an intra-agency agreement with HHS' planning and evaluation office to support a longitudinal evaluation study of the CBAE program to compare the impact of abstinence until marriage and comprehensive sexual education approaches on youth risk behavior.

Evidence: 1) Memorandum of Understanding with OPA for Website and Parent and Teen Guides (Agreement No. 05ACYFIA14); 2) Intra-Agency Agreement with OPA for National Abstinence Education Media Campaign (Agreement No. 06ACYFIA1); 3) Intra-Agency Agreement with OPA for Abstinence Until Marriage Grantee Evaluation Conference (Agreement No. 05ACYFIA19); and 4) Intra-Agency Agreement with ASPE for Evaluation of Adolescent Pregnancy Prevention Programs (Agreement No. 05ACYFIA6).

Does the program use strong financial management practices?

Explanation: The Department has established policy and prescribes a system for the administrative control of funds that complies with statutory requirements, and other Federal guidelines and standards. The abstinence education program office adheres to these policies and utilizes prescribed systems. The process begins with the development of a preliminary financial operating plan with estimates of the program's administrative expenses. This falls under the responsibility of the Deputy Associate Commissioner for the Family and Youth Services Bureau, the Division Director, and budget analysts. Final decisions on resource allocations are determined by senior ACF staff, who then submit the apportionment request to the Office of Management and Budget (OMB). After funds are apportioned and allotments received, funds are managed through a system of checks and balances to prevent violations of the Anti-deficiency Act. Abstinence education's program managers have provided an annual certification attesting to whether or not internal controls are in place and functioning as intended. Also, no material internal control weaknesses of the Abstinence Education program office have been observed. Strong financial management practices are also upheld for grantees. All non-Federal entities that expend $500,000 or more of Federal awards in a year are required to obtain an annual audit in accordance with the Single Audit Act Amendments of 1996, and OMB Circular A-133: Audits of States Local Governments and Non-Profit Organizations. A single audit combines the annual financial statement audit with additional audit coverage of Federal funds. The audit reporting package must be submitted to the Federal Audit Clearinghouse annually, no later than 9 months after the end of the organization's fiscal year. Entities expending less than $500,000 in a year are exempt from Federal audit requirements, but must make records available for review or audit by Federal agencies or pass-through entities (non-Federal entities from whom they received Federal funds), if requested. The National External Audit Review Center (NEARC) receives and reviews OMB Circular A-133 audits and other cross-cutting audits. As part of its review, NEARC may determine that audit findings should be brought to the attention of the HHS subagencies for appropriate action. This is accomplished by the issuance of NEARC Alerts, which indicate the nature of the problem and inform organizations that will be considered for placement on the Departmental Alert List. To date, ACF has not been alerted to audit issues that have emerged among CBAE or Title V grantees. However, if issues were to be identified, there are procedures in place for the Office of Grants Management and the program office to work with the grantees to take corrective action.

Evidence: 1) Statement of Auditing Standard (SAS-70) Auditors Reports - Program Support Center Operations; 2) FY 2005 HHS CFO Audit Opinion; and 3) Unified Financial Management System (UFMS) Business Case.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: To enhance program management, staff focuses on internal and external communications and program management tools such as the Employee Performance Management System (EPMS). Weekly staff meetings are held to discuss work priorities and to identify and troubleshoot problems. For example, some staff identified the need for more training on grant and contract administration. To address this, Grants Management and Payment Management Systems shared their expertise and provided training. The program also maintains a more formal way of tracking training needs through EPMS, which is used to review individual staff performance and to identify areas for further development. Communication is also maintained with our external partners. Regular meetings are held with contractors to ensure that objectives are being met. Similarly, program specialists monitor grantees regularly. The monitoring helps to identify problem grantees, which may result in site visits and/or technical assistance.

Evidence: The following discussion explains how program deficiencies are corrected using the system described above. For the CBAE grants review process, reviewers came from a pool of reviewers managed by program's bureau in FY 2005. For the FY 2006 grant review process, steps are being taken so that the abstinence education program office has more involvement in the management of the reviewer pool. This will result in more reviewers with subject-matter expertise specific to abstinence education. For the Title V program, there is a problem of shifting the application's due date. Steps are being taken for FY 2007 so that States will have to submit applications prior to the end of FY 2006 with sufficient time for review and modification, if needed. Other examples include the following: 1) distributing grant awards on a geographically diverse basis; 2) reducing the maximum award amount allowed from $800,000 to $600,000 in order to make more grant awards; and 3) requiring grantees to budget at least 15% towards program evaluation.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The program has no records of site visits made to Title V grantees while it was operated by the Health Resources and Services Administration. Since the program was transferred to the Administration for Children and Families in 2004, the program has scheduled planned site visits. Program specialists also monitor grantee activities through periodic phone calls, and annual progress reports where grantees provide information on major activities and accomplishments, identify challenges, and describe new activities planned for the next reporting period. The program staff also works closely with the grants management specialists to track actual expenditures and to verify that funds were used for their designated purpose. To do this work, the staff relies on the GATES system and SF-269A, which documents grantees' use of funds in eligible categories. The official grant files are maintained by the Office of Grants management and are accessible to program staff and managers. All files include grant award documents, application materials, budget information, reports, monitoring information and general correspondence.

Evidence: 1) ACYF/ FYSB; FY 06 Section 510 Abstinence Education Program Announcement, ACYF-FYSB-AE-01-06, CFDA#93.235; 2) Grants Administration Tracking & Evaluation System (GATES); and 3) Standard Form 269A: Financial Status Report.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program does not collect, compile, and disseminate grantee performance information in an accessible manner to the public. When the program was administered under HRSA, the program collected grantee performance data including: the rate of pregnancy to female teenagers aged 15-17; the proportion of adolescents who have engaged in sexual intercourse; the incidence of youths 15-19 years old who have contracted selected sexually transmitted diseases; the rate of births to female teenagers aged 15-17; and the unduplicated count of clients served, total encounters by clients, and communities served. Since the program was transferred to ACF, the program does not gather this information at the grantees level, except for the output data on clients served. The program has chosen to rely on national indicators, which cannot isolate the program and grantee's impact on the indicators.

Evidence: National indicators for the proportion of students in grades 9-12 who engage in sexual intercourse and the rates of births to unmarried teenage girl are founds in: 1) The Youth Risk Behavior Surveillance System (YRBSS); and 2)The National Vital Statistics System (NVSS).

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: All discretionary grants of the CBAE program are awarded through a clear competitive process that includes a qualified assessment of merit. Each year, a program announcement is issued to provide instructions for potential applicants. The program announcement is posted on www.Grants.gov, which is easily accessible to all public and private entities. In addition, the program office maintains the contact information for organizations that make inquiries on the abstinence education grants and conducts outreach by sending the program announcement once it is posted. After the program announcement is posted, the training and technical assistance contractor hosts several teleconferences to answer applicants' questions. Next, the criteria for grant awards are clearly stated for categories such as Objectives and Need for Assistance, Approach, Evaluation, Organizational Profiles, and Budget and Budget Justification. Applications are subject to panel reviews comprised of professionals with expertise in abstinence education, youth development, and social/human services. Applicants are informed that special consideration may be given to those applicants who meet the criteria for grant awards and target underserved geographic areas. Grant awards are made for multi-year project periods (no more than five years). In the first year of the project, 100% of the grants are awarded on a competitive basis through a peer review process. In fact, the program office received 446 applications and only 64 grants were awarded. In FY 2006, there were 500 applications and the program expects to award no more than 50 grants. Continuation grant applications are considered for subsequent years based on the availability of funds, satisfactory progress of the grantee, and a determination that continued funding is in the best interest of the Federal government.

Evidence: ACYF/FYSB; FY 05 Community-Based Abstinence Education Program Announcement, HHS-2006-ACF-ACYF-AE-0099, CFDA#93.010.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Abstinence Education program specialists monitor grantee activities through periodic phone calls and annual progress reports. Staff also works closely with the grants management specialists to track actual expenditures and to verify that funds were used for their designated purpose. To do this work, they rely on the GATES system and SF-269A, which documents grantees' use of funds in eligible categories. If problems are noted through the quarterly monitoring calls or progress reports, site visits can be arranged to gather additional information and implement corrective action which may include referring the grantee for technical assistance. For example, concerns were raised with a faith based organization's use of religious teachings in the abstinence education curriculum. The resulting recommendations from the site visit included: 1) making the website for the abstinence education program distinct from the church website; and 2) removing scriptural references from the abstinence education curriculum. Other noted concerns included the need for an employee policy and procedures manual and an accounting procedures manual. The program specialist is continuing to monitor this grantee closely to ensure that recommendations are adopted. The program office aims to complete 25 site visits per year and will be competing a contract to obtain the support needed to achieve this goal. Finally, official grant files are maintained by the Office of Grants management, which are accessible to program staff and managers. All files include grant award documents, application materials, budget information, reports, monitoring information and general correspondence.

Evidence: 1) Community-Based Abstinence Education Grant Program Quarterly Grant Status Update; 2) Community-Based Abstinence Education Grant Program Site Visit Monitoring; 3) Grants Administration Tracking & Evaluation System (GATES); and 4) Standard Form 269A: Financial Status Report.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The CBAE program collects output data, including: the number of youth served, the hours of service provided to each youth, and the number of youth that complete the program. CBAE grantees also report on the project outputs and outcomes that they have identified for themselves. However, this data is not compiled and disseminated to the public in an accessible manner.

Evidence: The program publishes data from the Youth Risk Behavior Surveillance System (YRBSS) and the National Vital Statistics System (NVSS) in the congressional justifications. However, these data are national indicators, which cannot isolate the program and grantee's impact on the nationwide trend. These national indicators include: 1) the proportion of students in grades 9-12 who engage in sexual intercourse; and 2) the rates of births to unmarried teenage girls.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The long-term measures show progress in decreasing out-of-wedlock births and lower rates of sexual activity among teens. For example, the national youth survey conducted by the Centers for Disease Control and Prevention show a decrease in the percentage of students who report that they have had sexual intercourse from 53.1 percent in 1995 to 46.7 percent in 2003. Also, birth rate per 1,000 women have declined from 43.8 in 1995 to 34.8 in 2003.

Evidence: 1) The Youth Risk Behavior Surveillance System (YRBSS); and 2) The National Vital Statistics System (NVSS)

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program has newly established program specific, annual measures. However, baselines will not be established until March 2009. As such, no data is available to show that the program is achievings its annual goals.

Evidence: 1) The Youth Risk Behavior Surveillance System (YRBSS); and 2) The National Vital Statistics System (NVSS)

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program is unable to demonstrate that management practices have resulted in quantifiable efficiency gains over the past year. However, the program has newly developed efficiency measures. It also notes that program funding has increased considerably from $20 million in FY 2001 to $113 million in FY 2006, without a corresponding increase in Federal staff administering the program. The program has standardized routine tasks and committed contracts to reduce the workload of Federal staff.

Evidence: The program's new efficiency measure is the cost of program delivery per youth. This is defined as follows: (FY 200X program budget) / (FY 200X # of youth served) = FY 200X cost per youth served.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Performance of the abstinence education program compares favorably to other programs with a broader scope. For example, below are two studies that show how an abstinence until marriage education program compares favorably to a reproductive health class where contraception information was provided. The NIH Office of Research on Minority Health and the National Institute of Child Health and Human Development supported a study in which six junior high schools in Washington D.C. were randomly assigned to the intervention or non-intervention control condition for a reproductive health class. Like many abstinence until marriage programs, there was a strong emphasis on strategies for resisting peer or social pressure to engage in sexual activity. Unlike abstinence until marriage education programs it differed by "providing contraceptive information and referrals to appropriate services for sexually active participants." At the end of the seventh and eighth grade (first and second follow-ups), intervention group females were more likely to report virginity. Another study examined the outcomes of a program funded by a CBAE grant. The curriculum covers eight curriculum units and emphasizes abstinence from a number of risky behaviors including sex. This study of middle and high school girls in Washington, D.C. showed that program participants were far less likely than their peers to have ever had sex. While their were significant differences in the breadth of program content, there were some similarities between the programs. For example, both programs targeted similar youth populations in Washington D.C. and both reported on the "Odds Ratio" contrasting the program participants to the control group. The results were comparable and show that with both approaches, program participants were more likely to delay the initiation of sexual activity. Also, the Odds Ratio grew stronger over time for participants in the CBAE study, whereas the NIH/NICHD study remained mre stable between seventh and eighth grade.

Evidence: 1) Aarons, S. J., Jenkins, R. R., Raine, T. R., El-Khorazaty, M., Woodward, K. M., & Williams, R. L. (2000). Postponing sexual intercourse among urban and junior high school students - a randomized control. Journal of Adolescent Health, 27(4), 236-247; 2) Lerner, R. (2004). Can abstinence work? An analysis of the Best Friends program. Journal of Adolescent and Family Health, 3(4), 185-192.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The Title V impact evaluation is being conducted by Mathematica Policy Research and uses a rigorous experimental design with random assignment of control and experimental group. The first year impact of the study was published in June 2005 and found that students in abstinence education programs were more supportive of abstinence and less supportive of teen sex than students in the control group. Students in the experimental group also had an increased awareness of the possible consequences of teen and non-marital sex than their counterparts in the control group. Later years' impact of abstinence education on student behaviors will be determined in later stages of the study. Although the program is in the planning stages of the community based abstinence education (CBAE) evaluation, there are findings from individual grantees' programs which do not meet the scope, quality, or independence criteria. Nevertheless, these evaluations reveal some interesting findings. For example, a grantee in the Panhandle region of Texas conducted a longitudinal study from 2004 to 2005 on its abstinence education curriculum and found that some effects of abstinence education programs were cumulative in that they grew stronger as teens remained in the program over time. The implication is that one-shot programs may yield very different results than continuous or ongoing programs. A grantee in Colorado compared state health data and surveyed teens to find that quality of teacher training and fidelity to curriculum can impact outcomes. Using qualitative methods, this study observed that clear boundaries, high expectations and follow-through on the part of parents were key attributes of adolescents who maintain sexual abstinence over 12 months.

Evidence: 1) Maynard, R. A., Trenholm, C., Devaney, B., Johnson, A., Clark, M. A., Homrighausen, J., & Kalay, E. (2005). First-Year Impacts of Four Title V, Section 510 Abstinence Education Programs. Princeton, NJ: Mathematica Policy Research Inc; 2) Rue, L. A., & Weed, S. E. (2005). Primary prevention of adolescent sexual risk taking: A school-based model. In A. Golden et al., (Eds.), Evaluating abstinence education programs: Improving implementation and assessing impact. Working papers and poster abstracts presented at the Abstinence Education Evaluation Conference: Strengthening Programs Through Scientific Evaluation, November 3-4, (pp. 61-74). Washington, DC: Department of Health and Human Services; and 3) Tanner, J. F., Ladd, S. (2005). Saturation abstinence education: An application of social marketing. In A. Golden et al., (Eds.), Evaluating abstinence education programs: Improving implementation and assessing impact. Working papers and poster abstracts presented at the Abstinence Education Evaluation Conference: Strengthening Programs Through Scientific Evaluation, November 3-4, (pp. 11-23). Washington, DC: Department of Health and Human Services.