Detailed Information on the
Strengthening Historically Black Graduate Institutions Assessment

Completing the study on the financial health of minority serving institutions of higher education and utilizing the results to validate program performance measures and improve program performance.

Developing strategies to use efficiency and performance data for program improvement purposes.

Making grantee performance data available to the public in a transparent and meaningful manner.

Measure: Graduation/Completion: The number of PhDs, First Professional, and Master's degrees awarded at HBGIs.

Explanation:

Measure: Enrollment: number of full-time graduate students enrolled at HBGI grantee institutions.

Explanation:

Measure: Cost per successful outcome: annual appropriation for given year divided by the number of doctorates, first professional, and masters degrees awarded in that year.

Explanation:

Is the program purpose clear?

Explanation: The purpose of this program is to provide financial assistance to Historically Black Graduate Institutions (HBGIs) to establish or strengthen their physical plants, financial management, academic resources, and endowments.

Evidence: Section 326 of Title III of the Higher Education Act (HEA) states that the Secretary of Education shall award program grants to 18 postgraduate institutions making a substantial contribution to the legal, medical, dental, veterinary, or other graduate education opportunities in mathematics, engineering, or the physical or natural sciences for Black Americans.

Does the program address a specific and existing problem, interest, or need?

Explanation: The Strengthening HBGIs program supports professional instruction in the physical sciences, engineering, mathematics, or other scientific disciplines in which African Americans are underrepresented. Congress explicitly authorized funding for HBCUs in 1965, in part, based on a finding that States and the Federal Government discriminated against Black institutions in the allocation of land and financial resources under the Morrill Act of 1862. Congress also found that the Black institutions had been discriminated against in the award of grants and contracts under the Higher Education Act and other Federal programs that benefit institutions of higher education and that the current state of Black colleges and universities is partly attributable to these discriminatory actions. While African Americans continue to lag behind their White cohorts in overall education attainment, HBCUs have contributed significantly to the effort to attain equal opportunity through postsecondary education for Black, low-income and educationally disadvantaged Americans. According to NCES statistics, in 2003, the immediate enrollment in college after graduating from high school was almost nine percentage points lower for black students (57.5%) than for white students (66.2%). Also, in 2003, full time degree-seeking black undergraduate students graduated from four-year institutions within six years at a rate that was 18.8 percentage points lower (38.5%) than the percentage for comparable white students (57.3%). In 2001-2002, African Americans earned only 9% of the bachelor's degrees, 8.4% of the master's degrees, and 5.4% of PhDs awarded in the United States, though African Americans comprise 13% of the population.

Evidence: Section 326 of Title III of the HEA; Section 321 of Title III of the Higher Education Act findings and purposes; NCES Digest of Education Statistics, 2003 (Tables 264, 267, and 270); Census 2000 Brief--The Black Population: 2000, Bureau of the Census.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: There are several Federal and private efforts including fellowships that provide funds/services to HBGIs and their students. President Bush signed Executive Order 13256 on February 12, 2002, (the fifth in a succession of executive orders designed to enhance federal funding for America's HBCUs) specifically to, among other things, increase opportunities for these institutions to participate in and benefit from Federal programs, as do other colleges and universities. While ED's Strengthening HBGIs program is designed to provide services to postgraduate institutions that make substantial contributions to the legal, medical, dental, veterinary, or other graduate education opportunities in mathematics, engineering, or the physical or natural sciences for Black Americans, related programs established in other agencies are significantly smaller in scope and only address purposes specific to their agency's mission. While the Strengthening HBGIs program and the Strengthening HBCUs program share similar goals and objectives the programs are designed to serve a different target audience. Strengthening HBGIs supports only graduate programs such as legal, medical, dental, veterinary, other graduate education opportunities in math, engineering, or the physical or natural sciences while Strengthening HBCUs program enhances the physical, financial and academic resources of undergraduate institutions. If an HBGI receives funding under this program, it cannot receive funding under the Title III Part A.

Evidence: Presidential Executive Orders 12232, 12320, 12677, 12876, and 13256. The statutory purposes for the HBCU program are provided in Title III, Part B, Sec. 321 of the HEA. The statutory purposes for the HBGIs are provided in Title III, Part B, Sec. 326(a).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: No major design flaws have been identified. The authorizing legislation strictly prescribes the funding formula and identifies the specific institutions eligible to receive program funding.

Evidence: Section 326(f) and (g) of Title III of the HEA.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program is well targeted to reach 18 eligible institutions defined as independent professional or graduate institutions and programs eligible for grants under Section 326(e). Further, funds are used to directly and effectively meet the program's purpose of providing financial assistance to HBGIs to establish or strengthen their physical plants, financial management, academic resources, and endowments. The statute provides for a wide variety of uses of funds in these areas depending upon individual institutional needs. Recent 2002-2003 annual performance reports indicate that 55 percent of expenditures were spent on activities related to academic quality, 16 percent on student services and student outcomes, 21 percent on institutional management, and 8 percent on fiscal stability.

Evidence: Section 326 of Title III of the HEA authorizes grants to Morehouse School of Medicine, Meharry Medical School, Charles R. Drew Postgraduate Medical School, Clark-Atlanta University, Tuskegee University School of Veterinary Medicine, Xavier University School of Pharmacy, Southern University School of Law, Texas Southern University School of Law and School of Pharmacy, Florida A&M University School of Pharmaceutical Sciences, North Carolina Central University School of Law, Morgan State University, Hampton University, Alabama A&M, North Carolina A&T State University, University of Maryland Eastern Shore, Jackson State University, Norfolk State University, and Tennessee State University. Section 326 also provides a list of authorized uses of funds. Grants are distributed by formula to all eligible HBGIs. 2002-2003 Annual Performance Reports.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The previous HBGI long-term measures track the percentage of institutional project goals that were successfully completed in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes. In conducting the PART review it was determined that these measures may not effectively measure program performance. Therefore, ED has replaced these measures with new outcome measures reflecting enrollment and graduation rates at HBGIs. These measures, as well as a persistence measure, have also been established for the Title III Strengthening Institutions Program, Strengthening HBCUs, Title V Developing HSIs, and Howard University resulting in consistent measurement across ED programs that focus on strengthening institution programs that serve large minority populations. IPEDS statistical data indicates that 20 percent of the professional degrees are earned by African Americans at HBGIs.

Evidence: FYs 2005 and 2006 Program Performance Plans. Data for the measures aggregated across all of the Title III and Title V programs and disaggregated by individual program are derived from the Title III electronic performance measurement system. Data for the new measures are derived from electronic annual performance reports for HBGI grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks. NCES Digest of Education Statistics, 2001, IPEDS "Completion" survey, 1999-2000).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Ambitious targets and timeframes have been established for ED's new measures that focus on enrollment and graduation rates at HBGIs using program data reported currently by institutions in IPEDS. These targets have been established considering program history at HBGIs, the comparison of the HBGI performance with the performance of all Title IV institutions and with similar programs at ED. HBGI program goals for these measures are projected through 2011. In addition, ED had set ambitious targets and timeframes through 2011 for its former long-term measures in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes.

Evidence: Please see measures tab.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The previous HBGI annual performance measures tracked the percentage of institutional project goals that were successfully completed. In conducting this PART review, it was determined that these measures may not effectively measure the performance of the overall program. Therefore, ED has replaced these measures with a new outcome measure reflecting graduation rates at HBGIs. This new measure, as well as a persistence measure, has also been established for the Title III Strengthening Institutions Program, Strengthening HBCUs, Title V Developing HSIs, and Howard University resulting in consistent measurement across ED programs that focus on strengthening institutions programs that serve large minority populations.

Evidence: Evidence/Data: FYs 2005 and 2006 Program Performance Plans. Title III Electronic Annual Performance Report. Data for the new measures are derived from electronic annual performance reports for HBGI grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: ED has established baselines and targets for its new measure focusing on graduation rates at HBGIs using program data reported by institutions in IPEDS. In addition, baselines and targets have been established for the program's current annual measures relating to improving academic quality, institutional management and fiscal stability, and student services and student outcomes of participating institutions.

Evidence: Please see measures tab.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: In response to findings and recommendations in the June 2000 Inspector General-issued audit on Title III programs and to better measure the success of Titles III and V, the Department conducted an extensive outreach effort in fiscal year 2001 to solicit recommendations for the Titles III and V program indicators and develop a new performance measurement system. The Title III and V grantee communities were informed and consulted in a number of ways, including program focus group discussions, meetings with project directors, regional grantee meetings, national conference calls, and outreach to college presidents and associations. The goals are based on information that all postsecondary schools already collect and use for a number of purposes. As part of the application process all partners commit to and work toward the specific goals of the program. Grantees must provide a five-year plan that describes their plans for accomplishing the goals and objectives of their specific projects. These projects are aligned with the purposes of the program and the program goals. Many grantee projects involve areas such as enhanced curricula, academic construction, academic success, technology, and educational materials, all of which are directly related to the new performance measures. In addition, annual performance reports are required for all grantees.

Evidence: Review of Title III Programs, HEA, Compliance with GPRA Requirements for Implementation of Performance Indicators (ED-OIG/A04-90014, June 2000). Office of Higher Education Programs Needs to Improve Its Oversight of Parts A and B of the Title III Program (ED-OIG/A04-90013, December 2000). EDGAR 34 CFR 74.51. Annual and final reports from the Title III Performance Measurement System collect data on academic quality, institutional management and fiscal stability, and student services and student outcomes. The Performance Measurement System for Title III is designed to acknowledge the diversity of institutions that are served, use institutional data from IPEDS for context, accommodate the diversity of institutional activities, and permit the aggregation of data in a standardized format.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Department is currently implementing a study on the financial health of institutions of higher education eligible to participate in the institutional assistance programs authorized by Titles III and V of the HEA. A purpose of the analysis is to determine whether the financial status of the institutions is improving or becoming worse and to identify what drivers are impacting the financial health of institutions. The analysis would also examine whether enrollment, persistence, and graduation are drivers of financial health. In addition, the analysis would show whether the programs authorized by the HEA are positively impacting the institutions financial health. Mathtech, Inc. conducted the last performance measurement study of ED's Title III Programs in 1995. The report presents the results of a survey of all 1995-96 Part A and Part B Title III grantees as well as in-depth case studies conducted at 19 institutions. The study indicated that institutions used Title III grants primarily to strengthen academic programs rather than on institutional management. The study also found that most Title III schools were not in severe financial difficulty.

Evidence: Data from the Study of the Financial Health of Institutions of Higher Education Eligible to Participate in the Institutional Assistance Programs authorized by Titles III and V of the HEA will be available in FY 2007. Performance Measurement Study of the U.S. Department of Education's Title III Strengthening Institutions Program (Mathtech, Inc., 1995): www.ed.gov/rschstat/eval/highered/title3perfmeas/index.html.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ED has not satisfied the first part of the question because program performance changes are not identified with changes in funding levels. The program, at this time, does not have sufficiently valid and reliable performance information to assess (whether directly or indirectly) the impact of the Federal investment. However, ED has satisfied the second part of this question in that ED's budget submissions show the full cost of the program (including S&E). ED's 05 integrated budget and performance plan includes the program's annual and long-term goals.

Evidence: NA.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ED has identified strategic planning deficiencies and taken meaningful steps to address these deficiencies. ED has refined its annual and long-term performance measures, and has added an efficiency measure. Further, ambitious annual and long-term targets have been established based on program and benchmarking information. ED's new performance measures are based on data already collected in IPEDS and provide for improved comparison of information across different programs. ED is also planning to review its current annual performance report and ensure that the information requested provides quality data and evidence of the success of the program, as well as of success in the individual projects.

Evidence: ED has created a new efficiency measure for the program and revised the annual and long-term performance measures to standardize the HBGI measures with performance measures for other similar programs within OPE. This will enhance strategic planning, reporting, and program management. Further, the data for the performance measures will come from IPEDS data collected by NCES, which increases the credibility and reliability of the data, as well as ensuring consistent data availability.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: ED requires grantees to submit regular and timely information via electronic means (i.e. annual performance reports, interim reports, and final reports). Information on grantee compliance with program requirements and objectives is collected annually and continuation awards are dependent upon a determination of substantial progress being made. The information collected is used to improve program performance and to provide technical assistance to grantees to assist them in meeting their proposed goals and objectives. In addition, ED has taken meaningful steps to address deficiencies highlighted in a recent GAO report entitled, "Low-Income and Minority Serving Institutions: Department of Education Could Improve Its Monitoring and Assistance" (GAO-04-961). GAO recommended that ED take steps to ensure that monitoring and technical assistance plans are carried out and targeted to at-risk grantees and that the needs of the grantees guide the technical assistance. One of those steps was to implement e-Monitoring, a new software tool that enables grant program staff to better monitor grants and assess risk. Program staff use this information to target technical assistance to grantees. Further, ED has revised its annual and long-term performance measures to more directly focus on outcomes and reflect the purpose of the program. ED uses program information to better manage the program and to make modifications, if needed, to improve program performance. In June 2005, ED met with partners to discuss regulating the definition of a program graduate. This action is in response to information learned about the wide variety of ways institutions define a "graduate."

Evidence: ED launched an electronic monitoring system (e-Monitoring) in December 2004. Program managers conduct comprehensive reviews of grantee interim reports and annual performance reports to ensure that performance measure standards are being met. Applications are reviewed in accordance with Federal statutes to ensure the planned activities are reflective of performance measures. Program monitors use interim, annual and final performance reports for targeted site assistance visits and monitoring visits as assurance that activity goals are completed as proposed or aligned to more effectively achieve intended goals. Actions such as high-risk designation leading to decreased funding, establishing special or additional reporting procedures or cancellation of grants may be imposed. Program staff places a hold on grantee's federal funds in cases of non-compliance or loss of accreditation. Further, staff approves or denies requests for budgetary changes that may affect the scope of approved projects. Based upon oversight information, institutions may have continuation funding reduced.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: While ED has initiated several efforts to improve accountability in its programs, it is not clear that Federal managers of this program are held accountable for program performance. Across the Department, ED is in the process of ensuring that "EDPAS" plans -- which link employee performance to relevant Strategic Plan goals and action steps - hold Department employees accountable for specific actions tied to improving program performance. ED is also revising performance agreements for its SES staff to link performance appraisals to specific actions tied to program performance. Finally, ED is reviewing its grant policies and regulations to see how grantees can be held more accountable for program results.

Evidence: The President's Management Agenda scorecard (Human Capital and Budget & Performance Integration initiatives) notes ED's efforts to improve accountability. The Department's Discretionary Grants Improvement Team (DiGIT) recommendations indicate that ED is reviewing its grant policies and recommendations.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Federal funds are obligated within the timeframes set out by ED schedules and used for the purposes intended. In FY 2006, ED plans to make the NCC awards by the end of April, this is two months earlier than in past years. This improvement gives institutions funding information sufficiently early to enable them to plan effectively and begin hiring processes. Further, to ensure that funds are properly managed, ED monitors grantees' financial records using GAPS and quarterly reports from the Office of the Assistant Secretary for OPE to alert staff to any grantees' excessive drawdown or inactivity of Federal funds. In addition, to ensure that funds are properly managed, ED has strengthened its financial management practices by implementing e-Monitoring. ED e-Monitoring is a new software tool that enables grant program staff to better monitor their grants and helps to prevent potential fraud, waste, and abuse.

Evidence: Title III Performance Measurement System; annual spending plans and program and partners' financial records; ED e-Monitoring was implemented December 22, 2004; and GAPS.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ED has developed an efficiency measure that is based on the number of students that successfully receive a PhD. The measure is determined by dividing the program's annual appropriation by the number of students receiving a PhD in that year. A similar measure (assessing the federal cost of the number of students graduating) is being established for the HSIs, HBCUs, and Title III A programs, as well as for Howard University. This change may enable ED to assess program performance across institutions with similar types of missions. In addition, ED has other procedures in place to achieve efficiencies and cost effectiveness. ED relies on competitive sourcing to provide technical and other administrative support. Contractual support provides for electronic submission of applications, as well as electronic submission of performance reporting, in addition to expert administrative and technical support and increases the cost effectiveness of the programs.

Evidence: Data for FY 2003 show that the Federal program cost per graduating student is $13,173. (FY 2003 appropriation for Strengthening HBGIs program: $53.4 million. Number of advanced degrees: 4,055). Multiple contracts provide database, technical assistance, and reporting support. Electronic applications and annual and interim performance reports also create efficiencies in data collection, analysis, and reporting.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Strengthening HBGIs program collaborates and coordinates effectively with other HEA Title III programs??Strengthening Institutions, Strengthening Alaska Native and Native Hawaiian-serving Institutions, Strengthening Tribally Controlled Colleges and Universities, and Strengthening HBCUs??and the Title V Developing HSIs program. If an HBGI receives funding under any of these programs, it cannot receive funding under the Strengthening HBGIs program, thus allowing resources to be managed more efficiently. Currently, these programs share common performance goals of improving academic quality, fiscal stability and institutional management, and student services and outcomes of eligible institutions and share common goals and data sources (IPEDS) now that the new measures of enrollment and graduation are in place. In addition, the Strengthening HBGIs program coordinates effectively with the TRIO programs, the Federal Student Aid programs, the Howard University program, the activities of the White House Initiative on HBCUs, the President's Board of Advisors on HBCUs, the National Association for Equal Opportunity in Higher Education, and the United Negro College Fund, as well as other ED programs with a strong emphasis on serving minority populations.

Evidence: To ensure coordination with other related programs, program managers hold meetings and have established common performance measures and goals, as well as parallel policies, administrative procedures and assessment tools. Additionally, National Project Directors' Workshops are held for these programs. Special rules outlined in Title III Sections 312(g), 316(d)(3), 317(e), and Title V Section 505 of the HEA.

Does the program use strong financial management practices?

Explanation: Because of recent audit reports that indicate ED's failure to adequately monitor grantees for their compliance with grant administration requirements, ED/OPE has implemented an e-Monitoring system enabling program staff to further mitigate against potential fraud, waste, and abuse. It allows staff to classify each grant into categories based on grantee performance, and maintains electronic records of phone, correspondence, and e-mail communications with grantees. Using ED's e-Monitoring and GAPS data along with other reports allows program officers to observe all funds applications made by an institution. Further, these systems are used to flag institutions determined not to have applied funds appropriately and to withhold funding, if necessary. ED continuously monitors grantee management and administration of Federal funds using GAPS and institutional financial reports.

Evidence: Low-Income and Minority Serving Institutions: Department of Education Could Improve Its Monitoring and Assistance" (GAO-04-961). All HBGI funded grantee institutions are documented in GAPS and e-Monitoring systems.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: For years, the Department of Education has utilized a paper collection instrument to obtain performance data and reports. ED has addressed this deficiency by replacing the paper collection instrument with an electronic performance reporting system to collect and report information more efficiently. In addition, each principal office must submit annual grant monitoring plans. Program offices must address the purpose, goals and objectives of the program; GPRA performance indicators and data; grantee performance; opportunities for technical assistance; and grantee or program risk factors. Furthermore, recent introduction of the e-Monitoring system has improved ED's ability to manage and monitor funds to further mitigate against potential fraud, waste, and abuse.

Evidence: The new Performance Measurement System for Titles III and V is designed to acknowledge the diversity of institutions that are served, use institutional data from the Integrated Postsecondary Education Data System for context, accommodate the diversity of institutional activities, and permit the aggregation of data in a standardized format. Further, ED has increased the number of on-site visits to institutions and has taken steps to improve the knowledge of staff conducting site visits through training initiatives. Section 6.4.4 of the Handbook for the Discretionary Grants Process; implementation of e-Monitoring; e-Grants; GAPS.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: New procedures have been developed for improving the monitoring of expenditures based on IG concerns. GAPS, e-Monitoring, annual and interim performance reports, financial status reports, and other internal reports are used to ensure that all related grant financial management practices are followed. Program officers provide continual oversight and coordination of activities of HBGIs via on-site technical and continuous electronic communication (email, fax, and telephonic).

Evidence: Review of Title III Programs, HEA, Compliance with GPRA Requirements for Implementation of Performance Indicators (ED-OIG/A04-90014, June 2000). Office of Higher Education Programs Needs to Improve Its Oversight of Parts A and B of the Title III Program (ED-OIG/A04-90013, December 2000). In addition to increasing efforts at on-site monitoring, program staff continues to review all performance reports that grantees are required to submit. In addition, program oversight includes documentation of grantees' use of funds, email communications, and project directors' meetings. This review process has resulted in the adjustment of grantee work plans that were not allowed within the scope of the program.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Performance data are collected annually through the IPEDS survey by NCES and through annual performance reports submitted by HBGIs to the program office. These data are available on the Department's website. Additionally, GPRA data are now reported in several formats, including on the ED website. In addition, ED has completed the first draft of the Strengthening HBGIs profile report. However, these data are not readily available to the public or on the internet at this time, and do not reflect program impacts.

Evidence: IPEDS data can be accessed at the College Opportunities On-line Website at http://nces.ed.gov/ipeds/cool.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Data for the antecedent long-term measures show that the program exceeded all of its annual targets in 2002 and two of its targets in 2003. Thus, the program appears to be on target to achieve its previously established long-term goals, which were projected through 2011. In FY 2002, grantees completed 88% of their academic quality projects, 78% of their institutional management and fiscal stability projects, and 86% of their student services and student outcomes projects. The three targets were set at 75%. In FY 2003, grantees completed 80% of their academic quality projects, 72% of their institutional management and fiscal stability projects, and 81% of their student services and student outcomes projects. The three targets were set at 75%. However, ED has replaced all of these measures with new outcome measures reflecting enrollment and graduation rates at HBGIs compared with that of all Title IV institutions. These measures, as well as a persistence measure, have also been established for the Title V Developing HSIs program, Strengthening Institutions program, Strengthening HBCUs, and Howard University. This change will result in consistent measurement across the ED programs that focus on strengthening institutions that serve large minority populations. HBGI program goals for these measures are projected through 2011. The early trend for program performance against these new measures is positive, as there was a significant increase in student enrollment from 2001 to 2002 (5.4%). Also, the HBGIs have increased their number of PhD's awarded (2.6%) during this same period -- (NCES, IPEDS, 2002).

Evidence: FYs 2005 and 2006 Program Performance Plans. Title III Electronic Annual Performance Report. Data for the newly proposed measures are derived from electronic annual performance reports for HBGI grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program (including program partners) achieve its annual performance goals?

Explanation: HBGI performance, as measured by the program's antecedent annual measures, has been generally positive. Grantees have exceeded their completion targets for three separate measures of activity. In FY 2003, grantees completed 80% of their academic quality projects, 72% of their institutional management and fiscal stability projects, and 81% of their student services and student outcomes projects. The three targets were set at 75%. In addition, adequate information is not available to determine the program's performance on its new annual measure, which reflects the graduation rate at HBGI grantee institutions.

Evidence: Evidence/Data: Data for the new measure reflecting graduation rates are derived from electronic annual performance reports for HBGI grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: An efficiency measure and baseline data were recently developed, but no targets or trend data are available to demonstrate improved efficiencies. ED has developed an efficiency measure that is based on the number of students receiving PhDs. The measure is determined by dividing the program's annual appropriation by the number of PhDs awarded in that year. Similar measures (based on the number of student's graduating each year have been established for the HBCUs program and the HSIs program, as well as for Howard University. This change will enable ED to assess program performance across similar types of institutions.

Evidence: Evidence/Data: Data for FY 2003 show that the Federal program cost per graduating student is $13,173. (FY 2003 appropriation for Strengthening HBGIs program: $53.4 million. Number of advanced degrees: 4,055).

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Although there are many federal, state, local, and private efforts that provide funding/services to HBGIs and their students that may be comparable, outcome data are not available on these programs to provide the basis for meaningful comparison. However, ED's establishment of similar performance measures for its programs that focus on strengthening institutions may eventually make comparisons of these kinds of programs possible.

Evidence: NA

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No independent evaluations of the Strengthening HBGIs program have been conducted recently. However, the Department is currently implementing a study on the financial health of institutions of higher education eligible to participate in the institutional assistance programs authorized by Titles III and V of the HEA. A purpose of the analysis would be to determine whether the financial status of the institutions is improving or becoming worse and to identify what drivers are impacting the financial health of institutions. The analysis would also examine whether enrollment, persistence, and graduation are drivers of financial health. In addition, the analysis would show whether the programs authorized by the HEA are positively impacting the institutions financial health. In October 1995, work began on a three-year performance measurement study of the U.S. Department of Education's Title III Strengthening Institutions Program. The primary purpose of this study was to document program accomplishments, and identify ways performance could continue to be measured and improved. Importantly, the study found that the Title III schools play an important role in serving students who have been historically underserved by postsecondary institutions.

Evidence: Data from the Study of the Financial Health of Institutions of Higher Education Eligible to Participate in the Institutional Assistance Programs authorized by Titles III and V of the HEA will be available in FY 2007. Performance Measurement Study of the U.S. Department of Education's Title III Strengthening Institutions Program may be found on ED's website at www.ed.gov/rschstat/eval/highered/title3perfmeas/index.html (Mathtech, Inc., 1995).