Detailed Information on the
Montgomery GI Bill- Veterans Education Benefits Assessment

Determine best methods to improve outreach efforts for education benefit programs; to be followed by the implementation of innovative tactics that are in step with mainstream advertising practices.

Determine the optimum level of monthly benefits required to accomplish the military recruitment and retention goals.

Create an outcome measure on veterans' readjustment to civilian life due to the benefit received in this program.

Reinstate a cost-effectiveness measure before the 2006 Budget, such as the 'Administrative Cost per Trainee' measure. Testing a measurement tool that measure cost effectiveness.

Measure: Percentage of Payments made accurately

Explanation:strategic target = 97%

Measure: Average number of days to complete original education claim.

Explanation:Completing educational claims in a timely manner is an essential part of our objective to provide the best customer service we can to beneficiaries while ensuring a smooth transition for veterans from active military service to civilian life. Measure is elapsed time, in days, from receipt of a claim in the regional processing office to the closure of the claim by issuing a decision. Original claims are those for first-time use of this benefit.

Measure: Average number of days to complete supplemental education claims.

Explanation:Completing educational claims in a timely manner is an essential part of our objective to provide the best customer service we can to beneficiaries while ensuring a smooth transition for veterans from active military service to civilian life. Measure is elapsed time, in days, from receipt of a claim in the regional processing office to the closure of the claim by issuing a decision. Supplemental claims are for subsequent periods of school enrollment.

Measure: Education Claims Completed Per FTE

Explanation:This measure determines the output generated by VBA Regional Processing Office Full-Time Employees (FTE) nationally and by office. The measure is calculated by using the following factors: 1. Completed work (Supplemental and original education claims tracked as completed end products - EPs) 2. Direct FTE from the presidential budget (PB) submission Formula : All End Products (EP's) (i.e., Actual Claims) completed in a fiscal year divided by Education's direct FTE as shown in the President's budget.

Is the program purpose clear?

Explanation: The Montgomery GI Bill (MGIB)-Active Duty program provides up to 36 months of education benefits. Benefits are payable for 10 years following release from active duty, and servicemembers contribute $100 per month during their first year of enlistment to be eligible. The MGIB-Selected Reserve (MGIB-SR) program is available to reservists and national guardsmen. Benefits may be used for degree and certificate programs, flight training, apprenticeship/on-the-job training and correspondence courses. They are payable for fourteen years, and no contribution is required. "Kickers" are added education benefits to enhance recruitment in "hard to fill" or critical skill areas. The primary purposes of these programs are clear. MGIB-Active Duty aims to aid in the transition from military to civilian life and in military recruitment. The primary purpose of MGIB-SR is recruitment, and the primary purpose of kickers are to fill critical positions. Congress, the Department of Veteran's Affairs' (VA) and stakeholders such as Veterans Service Organizations agree on these missions.

Evidence: The MGIB - Active Duty purpose is defined in Title 38, Part III, Chapter 30, Subchapter 1, Section 3001. MGIB - Selected Reserves and 'kickers' are defined in Title 10, Subtitle E, Part IV, Chapter 1606, Sec. 16131. These same purposes are echoed in a 2000 Klemm evaluation of MGIB.

Does the program address a specific and existing problem, interest, or need?

Explanation: It is widely accepted that MGIB aids recruitment into the armed services and with a service member's transition back to civilian life (a key concern in previous GI Bills, such as after World War II). There are a number of other recruitment incentives, as well, such as pay and benefits, enlistment bonuses, recruiters and advertising. It is unknown what effect MGIB has on recruitment, exclusive of these other incentives. "Kickers" are successful incentives for staffing hard to fill positions. In addition, MGIB-SR provides a recruiting incentive.

Evidence: 2000 Klemm Evaluation concludes that MGIB meets "some success" with transitioning, and is "successful" with recruitment (Chapters 30 & 1606). Kickers were not included in this evaluation as a separate component. Klemm studies also conclude that "the stated purpose that the MGIB-SR provide a recruiting incentive is ... being met."

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: A number of Federal, state, local and private efforts provide education benefits, including specific benefits targeting veterans. Similar programs in the Department of Defense provide education benefits (Loan Repayment and Tuition Assistance) and also serve as a recruitment incentive, like MGIB. While state benefits are varied, and may not have the purpose of being a 'readjustment benefit' they do target the same population and provide a similar service - education benefits. There is no similar program that provides as many options, such as payments for college classes, on the job training, or accelerated payments for high tech training.

Evidence: The website www.military.com lists education benefits available for veterans by state. In addition, the Department of Defense offers Loan Repayment Program and Tuition Assistance Programs. Other education benefits include federal loans, Pell grants, and scholarships.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: According to 38 U.S.C. 3014, MGIB is not intended to meet a specific level of educational benefits, but rather to 'help meet, in part,' expenses associated with higher education. A flaw in the program is that an 'optimum' level of benefits is not known. The benefit rates are set by Congress and except for specified rate increases in FY2001 - FY2004, the benefit rate will increase with the CPI. The CPI is not directly tied to increases in the cost of education. Stakeholders tend to measure the adequacy of its level against increasingly higher measurement tools. In addition, the effect of recent increases in monthly MGIB payments is unknown - while they may improve recruitment, they may simultaneously deter retention because the veteran usually separates from the military to receive the benefit. The most efficient level of monthly payment to accomplish and balance the program's goals (recruitment, transition to civilian life and retention) is unknown since these benefits are part of a complex and comprehensive package of pay and benefits.

Evidence: The most efficient levels of educational assistance monthly payment rate to support the program's purposes are unknown. Though these rates are established by legislation, and have increased approximately 78% ($528 to $985), it is unknown if a smaller rate increase would have also provided members the incentive to enlist in the military and provide an adequate level of educational assistance. Stakeholder measuring tools have varied from tuition at state schools to tuition, room and board at private schools.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The programs are effectively targeted based on the legislative purposes outlined in Titles 10 and 38. Only veterans who enroll in MGIB or MGIB-SR are served. In FY 2002, 93% of all education payments made by the VA accurately provided the correct amount to the right individual. Since the start of MGIB, 80% of those eligible have enrolled in the program. More than 59% had used some or all of their benefit by the end of FY 2002. (MGIB Biennial Report to Congress, Jan. 2003) A 1998 VA Inspector General Report stated "The quality review system is effective because it evaluates the accuracy of benefits awards."

Evidence: Since the start of MGIB, 80% of those eligible have enrolled in the program. More than 59% had used some or all of their benefit by the end of FY 2002. (MGIB Biennial Report to Congress, Jan. 2003) A 1998 VA Inspector General Report stated "The quality review system is effective because it evaluates the accuracy of benefits awards."

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: VA does have a multitude of strong long-term output measures designed to look at efficiency (i.e. the rate and quality by which claims are processed). Up to this year, VA considered MGIB Usage Rates an outcome goal for the program. This measure will be retained in its current form for the year; VA will simultaneously begin development of an outcome measure that evaluates the programs contributions towards successful readjustment to civilian life. DoD has long-term performance measures related to recruiting.

Evidence: VA's Performance Plan is contained in Volume 5 of VA's 2004 Budget. Page 54 outlines VA's education goals. DoD measures appear in mission statements and mission letters of each of the services, and in DoD's budget justification books for each branch of service.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: VA's targets for its long term output measures, such as the time to process and original or supplemental education claim, are ambitious. VA is projected to reach its target of a 97% payment accuracy rate in 2004. However, the program does not have sufficient outcome measures. A timetable for the new outcome measure is still in development. DoD has annual measures for both the quantity and quality of recruits.

Evidence: Volume 5 of VA's 2004 Budget states that VA aims to process an original education claim in 10 days, and any supplemental claim after the original claim in 7 days. For FY 2004, the goals are to process an original claim in 27 days and a supplemental claim in 12 days. DoD aims to recruit at least 90% high school graduates for the All Volunteer Force and Reserves.

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: Annual performance measures on timeliness (the time elapsed from when a veteran requests benefits to when they are received) and accuracy of payments contribute to VA's long term goal. These will help inform VA as they develop their long term outcome measure.

Evidence: Volume 5 of VA's 2004 Budget outlines these annual measures, and how they contribute to VA's strategic goals. DoD's budget justification books for each branch of service outline their annual goals.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: VA has baselines for its measures that serve as a starting point to compare improvements year to year. It is projected to reach its strategic target for accuracy of payments in 2004. Baseline and targets for the new outcome measure are in development. DoD also has baselines and ambitious targets for its annual measures.

Evidence: Baselines and targets are contained in Volume 5 of VA's 2004 Budget. DoD measures appear in mission statements and mission letters of each of the services, and in DoD's budget justification books for each branch of service.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: In order for a veteran to use their MGIB benefit for a program, it must first be certified by a State Approving Agency. These agencies are one of VA's most significant partners. State Approving Agencies, the Departments of Defense and Homeland Security, and the Veterans Advisory Committee on Education all work towards VA's education goals.

Evidence: State Approving Agency Contract language and the Veterans Education Advisory Committee charter demonstrate the commitment of these partners toward VA's goals.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: A comprehensive external evaluation of MGIB and MGIB-SR was undertaken by the Klemm Analysis Group in 2000. In addition, VA reports biennially to Congress on MGIB and MGIB-SR usage rates. In 2001, VA conducted an internal Survey of Veterans Satisfaction with the VA education benefits claims process.

Evidence: The Klemm Report, IG reports and the Principi Commission on Servicemembers and Veterans Transition Assistance are all independent evaluations of this program. VA has used the results of these evaluations to help set performance measures and identify areas of improvement.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: It is impossible to tell from VA's budget request what effect an increase or decrease in funding for program administration will have on achieving targeted goals. VA is able to demonstrate the link between their IT investments and quicker claims processing.

Evidence: VA's 2004 Budget submission does not tie the budget request to improvements in performance, with the exception of the planned education IT system.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: VA has baseline information, and a limited number of strong output measures that are regularly updated. VA has also recently devised a new, more revealing performance measure on usage rates and has begun work to create a more comprehensive outcome measures for the program.

Evidence: VA's 2005 Budget submission includes the new usage measure, and plans to create a new outcome measure. DoD has output measures for these programs, but no outcome measure.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: VA annually collects and updates performance measures on timeliness and accuracy to verify funds are obligated in a timely manner and spent for that intended purpose. VA also compares monthly actual usage to projected usage of the MGIB benefit. VA holds program partners accountable through this information collection. The funding allocated to VA regional offices and partners is dependent upon productivity levels. For example, if a State Approving Agency does not meet its required threshold for reviews, it will not get paid in full. If a regional processing center is not performing at an expected level, work is shifted to achieve the desired result. In addition, supervisors may receive additional training if they are not meeting their target.

Evidence: VA's 2004 Budget submission outlines increased program performance that will result from implementation of IT investment. VBA's Education Service Appraisals and conversations between OMB and VA on program management also demonstrate performance information is collected and used to better manage the MGIB program.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: MGIB Program managers at all levels are held accountable for program performance; good performance is rewarded with awards when possible. State Approving Agencies are held accountable for the approval and supervision of programs of education and training at educational and job training facilities which are approved for veterans training under the MGIB.

Evidence: The Education Director and all subordinate managers and employees have performance goals and objectives linked back to the program performance goals. Requirements for State Approval Agencies are outlined in Title 38, Chapter 36, Subchapter 1, and are evaluated annually. The Muskogee Regional Office received Tier II awards for achieving performance measure targets.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds for this program are spent on their intended purpose and comparing actuals to prior estimates is done on a monthly basis. In 2001, existing benefits were expanded, and VA's projections of the usage rates were higher than what has actually taken place. Due to the high estimates, VA has an unobligated balance in this area since 2001.

Evidence: VA's 2004 Budget submission, financial reports and internal tracking support this conclusion.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: While VA has an IT project in the planning stages to assist with reaching strategic targets and has measurements for efficiencies (i.e., time it takes to process a claim ), VA does not have such efforts to track cost effectiveness (i.e., the cost of processing one claim). VA does its best to reward superior performance on efficiency measures. For example, the Muskogee Regional Office received Tier II awards for achieving performance measure targets.

Evidence: There is no mention of cost effectiveness in the VA's 2004 Budget submission. VA used to report, per the 2001 - 2003 Performance Plans, an administrative cost per "trainee" (a veteran using the MGIB benefit) measure. This is no longer monitored.

Does the program collaborate and coordinate effectively with related programs?

Explanation: VA coordinates regularly with the Departments of Defense and Homeland Security, and is a member of the Department of Education's Federal Interagency Committee on Education (FICE).

Evidence: VA's Education Service is a designated FICE. FICE's primary responsibility is to contribute to make recommendations to ensure "effective coordination of federal programs, policies, and administrative practices affecting education programs." VA's Education Service participates and works with the Department of Labor on the initiative to modify time-based apprenticeship and on-job training programs to competency-based programs. VA is also working with Labor initiative to have State Licensing Boards and civilian apprenticeship sponsors recognize and give credit for apprenticeship and other on-job training that veterans receive while on active duty.

Does the program use strong financial management practices?

Explanation: VA was free of any material internal control weaknesses in this area. VA performs a statistically valid review of the payment accuracy of the four regional processing offices on a quarterly basis.

Evidence: Strong financial management practices are demonstrated in the Management Controls Process and identified in the VA Performance Plan and Audit Reports of 1999 and 2000. A VA Inspector General review of the education service quality review system concluded the education service had an effective quality review system.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: VBA conducts regular reviews of regional offices to ensure strong program management. Education Service confirms claims data through ongoing quality assurance reviews conducted on a statistically valid sample of claims. VA conducts a monthly review of all performance goals.

Evidence: The FY 2002 Performance and Accountability Report describes the quality assurance review process and the steps taken to improve the usage and processing of claims. The annual customer satisfaction surveys also provide direct feedback from the program participants.

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: VA makes annual progress towards its long term output goals, such as claims processing times and accuracy of payments. VA's long term outcome measure is in development.

Evidence: VA's 2004 Budget submission and Performance Plan demonstrate progress in achieving long-term goals; VA's 2005 Budget discuss creation of a new outcome measure.

Does the program (including program partners) achieve its annual performance goals?

Explanation: VA often achieves its annual performance goals. DoD often reaches its recruiting goals.

Evidence: VA's Performance plan and past budgets identify achievement of annual performance goals. DoD's budget justification books for each branch of service outline their annual goals.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: VA presents information on improved efficiency, but not on cost effectiveness. VA continues to improve its efficiency, as demonstrated by its recent IT system.

Evidence: VA's 2004 Performance plan and Budget request support this conclusion.

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation:  

Evidence: An attempt was made to compare the MGIB program to similar programs, however such a comparison is inherently difficult. There is no available data on DoD's Tuition Assistance or Loan Repayment programs, since they are part of larger package of recruitment benefits. The Americorps program, which is similar in that education benefits are provided in exchange for a service commitment, is new and has no data available. Programs in the Department of Education which help finance higher education are for different populations and different purposes, and have different measurement standards (i.e. use of national statistics).

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: The 2000 Klemm Evaluation concludes that MGIB meets "some success" to aid veterans in their readjustment, and that the program is "successful" with recruitment. The Klemm Evaluation also concludes that "the stated purpose that the MGIB-SR provide a recruiting incentive is ... being met."

Evidence: The Klemm Analysis Group did a comprehensive four volume evaluation of the program. The Klemm Group is a professional services firm that solves complex technical, policy and management problems by applying both conventional and innovative research methodologies.