Detailed Information on the
Troops-to-Teachers Assessment

Work in collaboration with the Department of Defense to strengthen collection of data on program performance and to relate program performance data to student achievement.

Address statutory obstacles that limit the program's ability to recruit participants to economically diverse high schools in need of highly qualified math and science teachers.

Examine data on State implementation of the program to determine if cost efficiencies can be achieved and establishing targets for the program's efficiency measure.

Measure: Percentage of participants who become mathematics, science, or special education teachers

Explanation:

Measure: The percentage of Troops-to-Teachers participants who remain in teaching for three or more years after placement in a teaching position in a high-need school district.

Explanation:

Measure: Percentage of participants who become teachers of record in high-need local educational agencies (LEAs).

Explanation:"Teacher of record" is defined as a Troops-to-Teachers participant who is hired by an eligible school district.

Measure: Recruitment cost per teacher of record. (New measure, added February 2007)

Explanation:"Recruitment cost" is defined as all overhead costs for the national headquarters and State offices, and "teacher of record" is defined as a Troops-to-Teachers participant who is hired by an eligible school district.

Is the program purpose clear?

Explanation: The purpose of the program is to assist eligible members of the Armed Forces to obtain teacher certification and become highly qualified teachers and to facilitate their employment in high-need schools and school districts.

Evidence: Section 2302(b) of the Elementary and Secondary Education Act, as amended by the No Child Left Behind Act of 2001

Does the program address a specific and existing problem, interest, or need?

Explanation: Numerous reports indicate that there is a shortage of highly qualified teachers to fill the nation's classrooms, especially in high-need school districts and public charter schools and in the fields of science, mathematics, and special education.

Evidence: The 1999-2000 Schools and Staffing Survey includes information on the percentage of students taught by teachers who are not certified or teaching "out of field" in subject-matter areas at the middle- and high-school levels for the 1999-2000 school year. At Middle Schools: In mathematics, 75 percent were taught by a certified teacher and 47 percent were taught by a teacher who had a major or minor in mathematics. Seventy-six percent of students taking science were taught by a certified teacher, and 59 percent were taught by a teacher with a major or minor in a science field. At High Schools: In mathematics, 83 percent were taught by a certified teacher and 82 percent were taught by a teacher with a major or minor. Eighty-five percent of students taking science were taught by a teacher certified to teach science and 86 percent were taught by a teacher with a major or minor in a science field.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: Many of this program's activities are similar to those in the Improving Teacher Quality State Grants and Transition to Teaching programs. However, the program provides a unique delivery mechanism that focuses on a special population that has a strong potential to become highly qualified teachers and stay in the classroom for many years.

Evidence: There are no other Federal efforts to recruit this unique group of potential teachers with significant subject-matter expertise.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Operationally, this program has succeeded in getting individuals into the classroom. However, it does not overcome design flaws in many States' alternative certification programs, which are not sufficiently streamlined.

Evidence: Program participants are dependent on State certification systems to obtain certification; many of these State certification systems require participants to meet burdensome requirements even when they are participating in alternative certification programs.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: This program is highly targeted on high-poverty districts and districts that have difficulty recruiting highly qualified teachers.

Evidence: Of the 5,079 Troops participants who have become teachers from 1994 to the present, 78 percent of those who received financial assistance and were hired as teachers are still teaching. In addition, about 84 percent are men, 37 percent are minorities, and 24 percent teach mathematics or science.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The two new long-term performance goals for this program are: 1) The percentage of program recuits who become highly qualified teachers and, 2) The percentage of Troops-to-Teachers participants who remain in teaching for three or more years after placement in a teaching position in a high-need school district.

Evidence: Baseline data is not yet available for these measures

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Department will develop targets and timeframes for performance measures once baseline data becomes available.

Evidence: ED cannot set reliable targets for these new long-term indicators without baseline data.

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: The two new annual performance goals for this program are: 1) The percentage of individuals recruited by Troops-to-Teachers who become "highly qualified" math and science teachers (per the No Child Left Behind definition) and, 2) The percentage of Troops-to-Teachers participants who remain in teaching for three or more years after placement in a teaching position in a high-need school district. The Department is working with OMB on developing an appropriate efficiency measure for this program.

Evidence: Baseline data is not yet available for these measures

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: While the Department has not yet established baselines for the annual measures, ambitious targets have been set as percentages above the baselines.

Evidence: ED cannot set reliable targets for these new annual indicators without baseline data.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: By law, the Department sends program funds to the Department of Defense, which obligates funds to the Defense Activity for Non-Traditional Education Support (DANTES). DANTES is working with the Department to continue collecting appropriate performance data about the program, including data to report on the PART indicators and for the required 2006 Report to Congress. DANTES has previously collected useful information about the persistence rates and demographic characteristics of Troops-to-Teachers program completers.

Evidence: DANTES targets recruitment on participants with skills and knowledge in high-need subject areas. While the PART measures for this program reflect a new set of metrics, they are still similar in spirit to the preexisting program focus: placing individuals with rich content knowledge in high-need classrooms.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Department is planning to conduct a survey of the Troops-to-Teachers and Transition to Teaching programs that will provide outcome data, but not data about educational impacts.

Evidence: Although this program is too small for the Department to conduct an evaluation of the program because of competing priorities, outside independent entities have occasionally conducted evaluations of the program.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ED has not satisfied the first part of the question because program performance changes are not identified with changes in funding levels. The program, at this time, does not have sufficiently valid and reliable performance information to assess (whether directly or indirectly) the impact of the Federal investment. However, ED has satisfied the second part of this question in that ED's budget submissions show the full cost of the program (including S&E). ED's 05 integrated budget and performance plan includes the program's annual and long-term goals.

Evidence:  

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Department will begin to track its new annual and long-term performance measures adopted through the PART process.

Evidence: ED and DANTES will begin collection of baseline data for the measures in the next year.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Department is working with DANTES to collect high-quality performance information and baseline information for the new performance measures. In addition, the program will develop implementation strategies based on its baseline performance data. The Department is working with OMB on developing an appropriate efficiency measure for this program.

Evidence: The Department is revising its agreement with DANTES to ensure that DANTES collects high-quality data on a regular basis.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: ED's managers are subject to EDPAS which links employee performance to relevant Strategic Plan goals and action steps, and is designed to measure the degree to which a manager contributes to improving program performance. To receive a "Yes," the ED needs to: (1) identify for OMB the federal managers for this program; and (2) demonstrate the relationship between these managers' performance standards and the program's long-term and annual measures; and (3) demonstrate the relationship between program partner's performance standards and the program's long-term and annual measures.

Evidence:  

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds are obligated within the timeframes set out by Department schedules and used for the purposes intended.

Evidence:  

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: This program has not yet instituted procedures to measure and improve cost efficiency in program execution. However, as part of the President's Management Agenda, the Department is implementing "One-ED" -- an agency-wide initiative to re-evaluate the efficiency of every significant business function, including the development of unit measures and the consideration of competitive sourcing and IT improvements. A "yes" answer is likely once the One-ED process is applied to this program's relevant business functions.

Evidence:  

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program collaborates and coordinates with other teacher quality programs in the Department.

Evidence: For example, the Teacher Quality Policy Group meets regularly to discuss teacher quality issues in programs authorized by the No Child Left Behind Act. In addition, the Department is in the process of developing common performance measures for teacher quality programs that includes Troops to Teachers.

Does the program use strong financial management practices?

Explanation: Recent agency-wide audits have not identified deficiencies in the financial management of this program.

Evidence:  

Has the program taken meaningful steps to address its management deficiencies?

Explanation: While major material internal management deficiencies have not been identified for this program, DANTES has implemented data-driven procedures to diagnose potential management problems.

Evidence: DANTES has established a data analysis process to maintain a close scrutiny of program activity. Program data related to the number of individuals registering, applications for financial assistance, funds obligated and expensed, and teachers hired are reviewed at least weekly. DANTES maintains a database containing information on every individual registering for the program, including contact information, personal information (gender, ethnic background, DOB, branch of service, rank, military job skill, years of service, etc), academic background, certification program enrollment, teaching assignments, financial assistance provided, fulfillment of teaching obligation, and counseling notes.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: By law, funds for this program are obligated to a single entity -- DANTES. However, DANTES then supports only eligible candidates from the military, as proscribed by the statute.

Evidence:  

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Depatment of Education program staff work closely with staff from DANTES and monitor DANTES's activities through annual reports that include performance data. In addition, DANTES funds 33 State support offices to assist participants with State certification requirements and employment leads in a total of 44 States. The State offices submit monthly activity reports and semiannual performance and financial reports to DANTES.

Evidence:  

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The Department collects performance data annually from DANTES but has not yet displayed this information to the the public in a meaningful manner.

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: It is too early to determine whether the program is achieving its long-term performance goals. The Department will establish the baseline in FY 2004.

Evidence:  

Does the program (including program partners) achieve its annual performance goals?

Explanation: It is too early to determine whether the program is achieving its annual performance goals. The Department will establish the baseline in FY 2004.

Evidence:  

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The Department is working with OMB on developing an appropriate efficiency measure for this program.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: While common measures are being explored for ED's teacher quality programs, this program serves a unique niche by exclusively recruiting retiring military personnel. When more performance information becomes available for other ED teacher programs, valid comparisons may be made on considerations of teacher persistence and qualifications.

Evidence:  

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: An independent evaluation of the program was conducted in 1998. Although the data are somewhat dated, it provides valuable information about the program. While the study contains no impact data, 71 percent of school administrators surveyed said that new Troops-to-Teachers staff were at least "above average" as compared to their other non-Troops first-year teaching colleagues.

Evidence: 'Profile of Troops-to-Teachers' by the National Center for Education Information. Authors: C. Emily Feistritzer, Michael D. Hill, and George G. Willett. A survey of Troops-to-Teachers participants in 1998, addressing their characteristics and attitudes toward teaching, teacher preparation, and teaching careers. Responses on a number of survey items were compared to responses of traditionally prepared public school teachers.