Detailed Information on the
Homeless Veterans Reintegration Program Assessment

Conducting a rigorous evaluation, to begin in 2007.

Continuing to improve cost effectiveness and reduce cost disparities across grantees through a competitive grantee selection process and through monitoring.

Strengthening accountability by instituting common performance measures that will allow comparison across various job training programs. Measures will track employment, retention, and earnings.

Continue to improve cost effectiveness and reducing cost disparities across grantees through a competitive grantee selection process and through monitoring.

Measure: Entered Employment Rate

Explanation:The Entered Employment Rate indicates the success of Homeless Veterans' Reintegration Program (HVRP) grantees' services in reintegrating homeless veterans into meaningful employment in the labor force. HVRP grantees collaborate with other care givers to address the many needs that homeless veterans have to overcome in order to become "job ready" again, which can include stabilizing their physical and mental health and gaining safe and affordable housing. The ultimate focus for HVRP grantees is improving vocational skills and job search abilities, resume writing, and overall employment skills, with the ultimate focus toward meaningful employment for the homeless veteran. The measure is a federal job training program common measure that enables comparisons of results among similar programs. The entered employment rate reflects the percentage of homeless veteran participants who obtained a job as a result of HVRP grantee intervention and is calculated by dividing the number of homeless veteran participants who entered employment by the number of homeless veteran participants who are enrolled in the program.

Measure: Employment Retention Rate

Explanation:The Employment Retention Rate indicates whether HVRP grantees' services lead to lasting employment for homeless veteran participants. The employment retention rate is intended to encourage quality assistance to homeless veterans. It is a Federal job training program common measure that enables comparisons of results among similar programs. The rate is derived by dividing the number of homeless veteran participants who are employed in both the second and third quarters after the exit quarter by the number of homeless veteran participants who exit during the quarter. This is the first year that this common measure has been applied to this program. The impact of this common measure on performance targets will be assessed, since it effectively increases the length of retention by an additional quarter from the previous measure.

Measure: Cost per Homeless Veterans Reintegration Program (HVRP) Placement

Explanation:This efficiency measure indicates grantees' ability to contain costs while at the same time increase the number of participants who obtain employment. The cost per homeless veteran placement is derived by dividing the funding provided to HVRP grantees by number of homeless veteran participants who enter employment.

Is the program purpose clear?

Explanation: The primary purpose of the Department of Labor's Homeless Veterans Reintegration Program (HVRP) is to assist homeless veterans to enter and retain employment. This competitive grant program emphasizes finding and retaining employment as a critical factor in eliminating homelessness or the threat of homelessness among veterans. HVRP was first created in 1987 under the Stewart B. McKinney Homeless Assistance Act (Public Law 100-77). The program was expanded under Public Law 107-95 (the Homeless Veterans Comprehensive Assistance Act) to include a demonstration for the purpose of determining the costs and benefits of providing referral and counseling services intended to prevent at-risk veterans, such as incarcerated veterans, from becoming homeless after their return to the non-institutionalized population. HVRP operates on the principle that securing meaningful employment enables self sufficiency and reduces veterans' homelessness.

Evidence: 1.PL107-95; http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=107_cong_public_laws&docid=f:publ095.107

Does the program address a specific and existing problem, interest, or need?

Explanation: According to estimates by the Department of Veterans Affairs (VA), 250,000 veterans are homeless on any given day. More than 500,000 individual veterans experience homelessness over the course of a year. The National Coalition for Homeless Veterans estimates that the VA only reaches only about 20% of those in need, leaving 400,000 veterans without supportive services. Veterans comprise about 23 percent of all homeless persons and one-third of the male homeless population. An estimated 45 percent of homeless veterans are capable of working, but need help finding a job. Homeless veterans face extraordinary barriers to obtaining or retaining employment; about 76 percent of them have alcohol, drug, or mental health problems. To find employment and become self sufficient, most homeless veterans require an array of integrated services. These services include substance abuse treatment, transportation, job referrals, and post-hire job coaching. HVRP integrates the supportive service resources funded by other agencies with employment-related services designed to help veterans prepare for employment, find a job, and retain that job over the long-term.

Evidence: 1. VA's veteran population estimate based on Urban Institute's analysis of a 1996 survey of homeless clients and providers conducted by the Census Bureau: Homelessness: Programs and the People They Serve, Technical Report, prepared by the Urban Institute for the Interagency Council for Homelessness, Sept. 1999.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: HVRP is the only nationwide Federal program that provides employment and training assistance to homeless veterans to fully reintegrate them back into the workforce. Traditional Federal-State job training and employment service programs are not well suited to assist homeless persons because they are mainly designed to serve people who are capable and willing to visit the offices where services are available, and who can readily be contacted by phone, mail, or computer about job referrals or training opportunities. While other targeted employment and training programs, such as the WIA Adult Program, the WIA Dislocated Worker Program, and the Employment Opportunities for Youth and Adults with Disabilities Program exist, HVRP's focus on employment and training services specifically targeted to homeless veterans is unique. Homeless veterans require substantial outreach efforts to identify them, to bring them into the training and employment service delivery system, and to coordinate the delivery of supportive services from various sources. HVRP grantees establish linkages to the local Workforce Investment Agencies and One-Stop Career Centers for the delivery of services to HVRP enrollees; typically that linkage requires the on-site presence at the HVRP location of an employment specialist funded by the Jobs for Veterans Act (JVA) Grants to States. HVRP grantees also operate in partnership with other programs for the homeless who provide shelter and address the problems of poor physical and/or mental health, and substance abuse. These partner programs are operated by the Department of Housing and Urban Development, the VA, and the Department of Health and Human Services. For example, the Department of Housing and Urban Development's (HUD) Homeless Assistance Grants program uses a shelter/housing approach to reduce homelessness in the immediate and short-term, but this program fails to implement an employment focus to increase the likelihood of a long-term change in an individual's life. In addition, the VA offers structured work opportunities and supervised therapeutic housing for at-risk and homeless veterans with chronic or severe physical, psychiatric and substance abuse disorders through their Compensated Work-Therapy (CWT) and Compensated Work Therapy/Transitional Residence Program to assist participants to become "work ready." These programs, however, do not focus on transitioning their participants into unsubsidized employment. In contrast, HVRP's emphasis on placing participants into meaningful jobs increases long-term employment retention and assists homeless individuals to regain their independence and self-sufficiency. HVRP grant recipients are required by VETS to leverage community, faith-based, and other resources in the design of individualized employment plan for each enrolled veteran, and to use a case management approach, commonly referred to as a "continuum of care", to coordinate the delivery of services. This holistic approach ensures that HVRP program services do not duplicate other programs' efforts, but instead coordinate the delivery of a broad spectrum of necessary services from a variety of entities.

Evidence: 1.) Planning Instructions through Federal Register Notice Issuance of HVRP Grants for Program Year (http://www.gpo.gov/su_docs/fedreg/a060420c.html#Veterans%20Employment%20and%20Training%20Service 2.) Pub. Law No. 107-95, Section 5 under the Homeless Veterans Comprehensive Assistance Act of 2001 (HVCAA).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The employment focus of HVRP takes advantage of the long-standing experience of the Labor Department in providing employment services. Each grantee is assisted by a State employment specialist funded by one of Labor's grant programs. The services provided by VA and HUD are necessary for the housing and medical care that the homeless require, but these agencies lack the employment focus and the linkages to the nationwide system of One-Stop Career Centers. VETS uses a competitive grant process to assure that HVRP grantees have the experience and skills to manage and coordinate the broad range of services from multiple organizations required to help the homeless secure long-term, sustainable employment. Because the initial grants awarded under HVRP are highly competitive, grants are awarded to the most capable, efficient, and cost-effective grantees. Grantees are generally non-profit, community-based organizations. Grants are awarded for one year with two additional years of funding possible. HVRP grantees may obtain funding for the additional years only by attaining or exceeding 90% of planned goals. The three-year competitive grant award design results in continuous process improvement as grantees gain experience. Furthermore, the annual competitive award cycle enables the VETS, as the grantor agency, to direct funds to areas with the greatest need and where there are service providers with proven ability to achieve the program purpose.

Evidence: 1. SGA Planning Instructions through Federal Register Notice Issuance of HVRP Grants for Program Year (PY 2004 & PY 2005)

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: All HVRP funds (other than minimal overhead allowances) must be spent on services to directly and positively impact homeless veterans' employment status. Administrative controls are designed to ensure that all HVRP expenditures are for that purpose. HVRP grants are awarded to urban and non-urban applicants to ensure that resources are geographically dispersed each year to areas that have significant populations of homeless veterans, and to grantees that have demonstrated ability to leverage non-HVRP resources to augment and complement their employment-related service activities. In addition, the grant award criteria provide weighted scoring incentives that favor applicants who demonstrate an ability to reach and serve "chronically homeless veterans or those veterans that have experienced significant periods of homelessness." VETS also works with the National Coalition of Homeless Veterans to provide technical assistance and guidance to potential grantees, which helps ensure that the best potential grantees are identified and encouraged to apply for HVRP funding. Grantees must use other-than-HVRP resources to obtain supportive services for enrolled veterans such as transportation or medical/dental services that are essential but are not actually employability development, skills training, or job finding activities.

Evidence: SGA Planning Instructions through Federal Register Notice Issuance of HVRP Grants for Program Year (PY 2004 / PY 2005).

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: HVRP has two specific long-term performance measures: Entered Employment Rate and Employment Retention Rate. Both of these measures ensure that the program focuses on outcomes that meaningfully support and demonstrate the purpose of the program. The Entered Employment Rate indicates the percentage of homeless veterans who obtained a job after being enrolled in the HVRP. To track participants' ability maintain employment over time, VETS measures the Employment Retention Rate, which indicates the percentage of participants who entered employment under the Program and who have retained employment for over six months.

Evidence: FY 2005 Performance and Accountability Report. http://www.dol.gov/_sec/media/reports/annual2005/ 2.) SGA Planning Instructions through Federal Register Notice Issuance of HVRP Grants for Program Year (PY 2004 / PY 2005). NOTE: Starting in 2006, HVRP will modify its traditional performance measures to fully comply with the Common Measure Definitions used by other Federal employment and training programs. After new baselines are established, this change should improve DOL's ability to compare HVRP performance outcomes against results generated within other government-wide employment and training programs.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: VETS has sought to adopt ambitious targets for both of its long-term outcome measures, and targets increase annually through 2011. Since the level of program funding increased dramatically - from $9.6M in FY 2000 to $17.6M in FY 2001 -- VETS was uncertain how this expansion in appropriations would impact program performance. Consequently, VETS established a strategy to set ambitious, yet sustainable performance improvement targets that take into account actual performance results. Over the past three years, VETS has increased its entered employment rate targets each year. For example, in PY 2003, VETS established an Entered Employment target for HVRP of 54.5% based on its performance in PY 2002 , while the actual PY 2003 result ended up being close to 61%. To reflect this performance outcome improvement, in PY 2004 VETS established an Entered Employment target of 60%, and achieved an actual performance outcome of 65%. In PY 2005, VETS increased the Entered Employment target again. As a result of these continual increases in targets, the Entered Employment goal increased by over 10% during three program years. VETS has taken a similar approach to setting performance targets for its goals for Employment Retention. VETS staff has some concerns about being able to demonstrate and achieve the same degree of performance outcome improvement over the longer-term because of the inherent difficulties in successfully serving chronically homeless veterans. Yet, despite these concerns VETS continues to establish ever increasing long-term Entered Employment and Employment Retention goals and targets through PY 2011. HVRP has targeted over a 13% increase in its Entered Employment Rate target between PY 2003 and PY 2011, with a PY 2011 target of 68%. HVRP's Employment Retention also is targeted to increase between PY 2005 and PY 2011, with a PY 2011 target of 62% representing an overall 4% increase in actual annual performance.

Evidence: 1.) Supplementary Evidence Table showing VETS' Long-Term Targets; excerpt from FY2005 DOL Performance and Accountability Report. http://www.dol.gov/_sec/media/reports/annual2005/ 2.) USDOL Strategic Plan for Fiscal Years 2007 - 2011. http://www.dol.gov/_sec/stratplan/strat_plan_2003-2008.htm 3.) Performance Goal DOL-07-1.1F; NOTE: The adoption of Common Measures for Federal job training and employment programs and the associated changes in some definitions and data sources means that new baselines for the HVRP outcome measures will be established in 2006. If the performance data indicate that the current projections are too conservative, VETS will revise upward the out-year targets based upon the actual performance results generated.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: As one of the programs using the job training common measures, HVRP's annual and long-term performance outcome measures are the same: Entered Employment Rate and Employment Retention Rate. Because the annual and long-term performance measures are the same, meeting each annual performance target ensures that the program is on track to meet or exceed its long-term performance goals.

Evidence: 1.) FY2005 DOL Performance and Accountability Report. http://www.dol.gov/_sec/media/reports/annual2005/ 2.) VETS Operations and Programs Activity Report (VOPAR).

Does the program have baselines and ambitious targets for its annual measures?

Explanation: As stated in answer to Q. 2.2, the program has an established baseline for each of its two outcome measures and associated annual targets that indicate commitment to continuous improvement. The baseline for Entered Employment Rate, established in 2003, is 54 percent, while the baseline for Employment Retention Rate, established in 2004, is 58 percent. VETS has targeted increases for both measures through 2011.

Evidence: 1.) FY2005 DOL Performance and Accountability Report. http://www.dol.gov/_sec/media/reports/annual2005/ 2.) VETS Operations and Programs Activity Report (VOPAR).

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The HVRP requires direct partnerships between VETS and the program grantees selected through the competitive award process. Because there are no intermediaries, the signed grant agreements signal absolute commitment to the annual and long-term goals of the program. Each grantee must submit and obtain VETS' approval of Annual Performance Goals which are then entered into an automated reporting system. Grantees' performance results are reported on a quarterly basis. The program management information system automatically informs VETS when any grantee's performance puts it in jeopardy of not meeting its annual target (falls short of 85% achievement of a target). In such cases, grantees must develop and submit for VETS' approval a Corrective Action Plan (CAP) that identifies actions the grantee will take to ensure goal achievement by the end of the performance period. When an appropriate corrective action plan is in place, VET program staff provides technical assistance as needed and performs monthly rather than quarterly reviews of the grantee's performance until performance either rises to acceptable levels, or until the grant is terminated.

Evidence: 1. Special and General Grant Provisions and grant agreement signature pages.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: VETS has not yet conducted a rigorous, comprehensive, independent evaluation of HVRP's effectiveness. However, VETS has used internal and on-going management assessments, on-sight monitoring visits, desks audits, and studies by the Government Accountability Office (GAO) as the basis for implementing program improvements. GAO examined HVRP at the request of the Subcommittee on Economic Opportunity, Committee on Veterans' Affairs, House of Representatives, as part of the subcommittee's consideration of HVRP reauthorization. GAO looked at (1), DOL's expenditures on HVRP Grants; and (2), measures and goals for assessing the effectiveness of HVRP. VETS established a baseline for its Employment Retention Rate partly as a result of recommendation from this study. VETS hopes to obtain an independent evaluation of the program using DOL resources dedicated to program evaluation activities in 2007-08.

Evidence: 1. GAO: Testimony Before the Subcommittee on Economic Opportunity, Committee on Veterans' Affairs, House of Representatives, Homeless Veterans: Job Retention Goal Under Development for DOL's Homeless Veterans' Reintegration Program (HVRP).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: VETS presents the resource needs for the HVRP completely and transparently in its annual budget requests. VETS uses empirically-based formulas that reflect past experience to estimate the relationship among veteran outcomes, grantee allocations, and budgetary resources. Budget request documents explicitly relate dollars to performance outcomes. VETS' 2007 program budget request of $21.838 million is a small increase over FY 2006's appropriation of $21.780 million. The requested funds are projected to support 88 grants that collectively will enroll approximately 14,100 homeless veteran participants, a slight increase over the 13,865 homeless veterans enrolled in FY2006. The unit cost for each participant is projected to decrease from $1,571 in FY2006 to $1,568 in FY 2007; the unit cost for each individual placed in a job is projected to decrease from $2,125 in FY 2006 to $2,100 in FY 2007. These measures demonstrate VETS effectiveness at projecting future resource needs based upon achieving program goals.

Evidence: 1. Supplementary Evidence Document - The VETS FY 2007 Congressional Budget Justification.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The HVRP addressed a significant strategic planning deficiency identified in a VETS internal review in PY 2004 which indicated that VETS needed to put a greater emphasis on employment retention for both program management and oversight purposes. HVRP implemented the Employment Retention Rate measure in 2004. With this measure in place, the program can demonstrate to DOL and Congress the program's long-term impact on homeless veterans. Also in PY 2004, VETS moved from a spreadsheet-based management information system that relied on multiple manual inputs of grantees' data and suffered from timeliness problems to an automated, real-time on-line reporting system: VETS' Operations and Program Activity Report (VOPAR). The implementation of VOPAR successfully addressed a strategic planning deficiency caused by the inability of the old data gathering process to deliver the timely and accurate program information needed for strategic planning and budget formulation.

Evidence: 1. GAO: Testimony Before the Subcommittee on Economic Opportunity, Committee on Veterans' Affairs, House of Representatives, Homeless Veterans: Job Retention Goal Under Development for DOL's Homeless Veterans' Reintegration Program (HVRP). 2.) VETS Operations and Programs Activity Report (VOPAR). 3.) USDOL Strategic Plan for Fiscal Years 2007 - 2011. http://www.dol.gov/_sec/stratplan/strat_plan_2003-2008.htm

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Performance data collected by VETS is both timely and credible, and it is used to manage and improve performance. VETS staff who serve as Grant Officer Technical Representatives (GOTRs) are stationed in each state. GOTRs routinely conduct oversight visits to the HVRP grantees to observe program operations and examine records, and monitor progress towards attaining outcomes. Every grantee sends to the responsible GOTR a standardized quarterly performance report, which is reviewed and verified within 45 days following the end of each quarter. Those data are entered into an automated system (VOPAR) by the grantee according to designated performance measures and targets specified in the grant award agreement and SGA. Any grantee that is not within 15% of an established goal is automatically identified. When this occurs, a Corrective Action Plan process is initiated and must be approved by the GOTR and VETS National Office staff. In PY 2005, VETS raised the threshold for obtaining second- or third-year funding from 85% to a minimum achievement level of 90% of the established goals specific to each grant. VETS' program reviews have indicated that grantees' performance usually improves with experience.

Evidence: 1.) SGA Planning Instructions through Federal Register Notice Issuance of HVRP Grants for Program Year (PY 2004) 2.) VETS Operations and Programs Activity Report (VOPAR). 3.) Special Grant Provisions/Final Agency allocation spreadsheets PY 2004.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and program partners have to meet prescribed performance standards that directly relate to HVRP performance goals and targets. Selected VETS staff at Regional and State office levels and members of the VETS' National Competitive Grants Expert Cluster (CGEC) are held accountable to individual performance standards that relate specifically to the HVRP annual performance. For example, during each program year CGEC members are required to develop a list of approved work items and a method of tracking activities; a performance element, and standards to evaluate their performance. Also, a member of the CGEC is participates in their annual performance appraisals. HVRP grantees are responsible for ensuring that their sub-grantees and/or program partners comply with the performance goals and targets established when the grant was awarded (or subsequently modified). For example, veterans employment specialists who are assigned to work with HVRP grantees are subject to having specific goals established for their performance standards that are consistent with both HVRP and JVA program goals. Grantees are required to report outlays, program income, and other financial information each quarter. Grantees must submit a Quarterly Technical Performance Report and a Quarterly Financial Report 30 days after the end of each Federal fiscal quarter to their GOTRs or to the State Director of Veterans' Employment and Training. These reports contain a comparison of actual accomplishments to established goals for the reporting period and any findings related to monitoring efforts. Data from these reports and other sources are used to hold individual managers and program partners accountable for their performance.

Evidence: 1. SGA Planning Instructions through Federal Register Notice Issuance of HVRP Grants for Program Year (PY 2004 / PY 2005) 2. VETS Operations and Programs Activity Report (VOPAR) 3.) Special Grant Provisions/Final agency allocation spreadsheets PY 2005 4.) Director's Memorandum #06-06 - National Expert Cluster Workgroups.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: HVRP grant funds are obligated by VETS in a timely manner and are administered in accordance with Federal guidelines. New grantees are selected and announced publicly on an annual basis as long as funds are available. They are sent a copy of the grant award letter, the Grant Provisions and Special Grant Provisions. Once a grantee provides to DOL the required information, as indicated in the Special Grant Provisions, the grantee can draw its allocation within 48 hours. In accordance with the Special Grant Provisions, each State is restricted to the amounts and purposes indicated by the grant award. Within 45 days following the end of each quarter, grantees account for the funds they drew down and expended using standardized reporting. Financial reports are reviewed, analyzed and posted to the Internet-based VOPAR system. VETS monitors fund use to ensure grantees' spending is consistent with their approved plans. Just $12,469 (.0007%) of the FY 2004 appropriation of $18,888,000 remained unobligated at the end of the fiscal year. The VETS also complies fully with key financial management statutes. Auditors have reported no findings of material weaknesses under the Federal Managers' Financial Integrity Act (FMFIA). Management controls by the DOL Administrative Finance office ensure that DOL does not obligate more funds than available, and that no violations of the Anti-Deficiency Act occur in this program.

Evidence: 1. OMB Circulars A-102 and A-87. 2.) VETS Operations and Programs Activity Report (VOPAR)

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The HVRP implements numerous procedures to measure and achieve efficiencies and cost effectiveness. These procedures involve both the new grant award and evaluation process, which uses projected targets and goals, as well as the determination of optional second and third year grant funding, which is based significantly upon actual performance outcomes. To ensure overall operational effectiveness and efficiency, HVRP also measures and incorporates the Average Cost per Placement in the decision to award second and third year funding. To receive second and third year funding, grantees must demonstrate and achieve actual performance outcomes that are within 90% of their projected goals. For example, grantees must demonstrate that their actual Average Wage at Placement is within 90% of their projected goal or have an approved corrective action plan that establishes procedures to ensure that grantee projected goals will be achieved to receive either second or third year funding. If either of these conditions are not met, VETS may withhold funds (once a systematic determination is made after a review of the grantee's actual performance and submitted corrective action plans). HVRP emphasizes the Average Cost per Placement measure because it illustrates outcome performance efficiency that is directly tied to the purpose of the program. The new grant award and evaluation process that was implemented for the first time in PY 2004 also demonstrates VETS commitment to measuring and achieving improved efficiency and cost effectiveness within the program. After all applications are rated and ranked according to the criteria specified within the SGA, the VETS national program manager assesses the potential awardees with respect to their projected Average Cost per Placement. While it is not practical to limit each potential grant recipient to the cost effectiveness benchmark set forth in the SGA, due to the variations in costs of living among all of the urban and non-urban areas of the country, the VETS national program manager (in collaboration with the DOL Grant Officer) calculates the Average Cost Per Placement for the entire group of potential awardees. The final selections for funding are made by disqualifying those potential awardees whose proposed Average Cost per Placement drives up the overall group's average too far above the benchmark. The result is an effective control of costs related to the primary outcome of the program.

Evidence: 1. SGA Planning Instructions through Federal Register Notice Issuance of HVRP Grants for Program Year (PY 2004) 2. VETS Operations and Programs Activity Report (VOPAR).

Does the program collaborate and coordinate effectively with related programs?

Explanation: The HVRP collaborates extensively with other programs. At the national level, VETS staff participate on a workgroup that is an extension of the United States Interagency Council on Homelessness. This group includes members from the Departments of Commerce, Defense, Education, Health and Human Services, Homeland Security, Housing and Urban Development, Interior, Justice, Labor, Transportation, Veterans Affairs, the General Services Administration, the Social Security Administration, Postal Service, the USA Freedom Corps and the White House Office of Faith Based and Community Initiatives. This high level collaboration and coordination ensures that HVRP resources are focused on providing employment and training related services, while at the same time assures that other agencies provide necessary supportive services. These strategic commitments are reflected in numerous memoranda of understanding (MOUs) between HVRP and other service providers necessary to support the "continuum of care" that homeless veterans need to prepare them for the employment and training opportunities provided under HVRP. These MOUs help to eliminate duplication in service delivery. For example, HVRP grantees coordinate extensively with local VA Hospitals to ensure that all homeless veteran participants receive the necessary mental and physical health services and substance abuse services required to enable them to enter into productive employment and training placements. At the Regional and State levels VETS staff ensure that coordination occurs between HVRP grantees and the State Workforce Investment System and the local One-Stop service centers where veterans can receive priority for service from any Department of Labor-funded employment and training program. At the service delivery level, grantees are required to demonstrate their ability to complement their HVRP-funded employment and training services by using a structured case management approach that provides access to a wide variety of supportive services offered by providers outside of HVRP funding. Grantees are expected to, and in fact do, effectively gain access for program participants to other programs or services such as: food/clothing; provision of or referral to temporary, transitional, and permanent housing; referrals to medical and substance abuse treatment; and transportation assistance.

Evidence: 1. SGA Planning Instructions through Federal Register Notice Issuance of HVRP Grants for Program Year (PY 2005)

Does the program use strong financial management practices?

Explanation: VETS management practices comply fully with key financial management statutes. There has never been a finding of a material weakness in the financial management of the program under the Federal Managers' Financial Integrity Act (FMFIA). Any audit exceptions identified through independent auditors under the Single Audit Act (29 CFR, Part 96) are reported through VETS and reconciled by the Grant Officer prior to or during grant close-out. DOL's Office of the Assistant Secretary for Administration and Management provides the independent Grant Officer function, ensuring strong controls over grant awards and modifications, and its Administrative Finance Office controls the obligation of funds. VETS uses the U.S. Department of Health and Human Services (HHS) Payment Management System (HHS/PMS) for tracking funds provided to grantees through a secure system that ensures that only authorized grantee staff have access to program funds.

Evidence: 1. SGA Planning Instructions through Federal Register Notice Issuance of HVRP Grants for Program Year (PY 2005)

Has the program taken meaningful steps to address its management deficiencies?

Explanation: An example of a previous critical HVRP management deficiency was the lack of timely and accurate program information from grantees. This problem was caused by a management information system that was in use before 2004. The old system could not provide timely data with which VETS could make sound management decisions. Data were always too late to be truly valuable for either program management or even for strategic planning purposes. This deficiency has now been remedied. In PY 2004, VETS developed and implemented an on-line, fully automated reporting system called VETS Operations and Programs Activity Reports (VOPAR), thus removing the inherent reporting deficiencies and improving the overall quality and timeliness of the data and consequently, program management processes.

Evidence: 1.) VETS Operations and Programs Activity Report (VOPAR). 2.) FY2005 DOL Performance and Accountability Report. http://www.dol.gov/_sec/media/reports/annual2005/ 3.) USDOL Strategic Plan for Fiscal Years 2007 - 2011. http://www.dol.gov/_sec/stratplan/strat_plan_2003-2008.htm

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: All HVRP grants are awarded on a competitive basis. No sole-source awards have been allocated. VETS announces the availability of competitively awarded grant funds in published Solicitations for Grant Applications (SGAs) that are posted on-line at www.fedgrants.gov and also are printed in the Federal Register. A posted SGA outlines the requirements of the program and informs the potential applicants of the specific information and forms required to apply for the funds. Posted SGAs clearly identify the terms of the grant as being one year with two option years. However, all option year allocations are contingent upon a strict adherence to their performance goals. For example, in PY2004 two grantees who applied for option year funding were denied their request because their performance did not meet the proposed goals specified within their competitively awarded base year grant. More than half of the PY 2006 grant recipients were first-year recipients. VETS also actively seeks out and provides incentives for new organizations to apply for HVRP funds. For example, a recent PY 2006 SGA targeted eligible applicants that had not previously received HVRP grants or other competitive grants from VETS. Once the SGA is announced to the public, potential grantees usually have 30 days to submit an application for Federal funding. Before the grant application deadline, VETS, in coordination with the Grant Officer, selects the grant review team members (usually comprised of VETS national office and field staff, including those individuals serving on the CGEC) who evaluate the applications that meet the initial review requirements specified by the Grant Officer. Once all applications are received and filed, the grant review team rates, ranks, and scores all of the applications and makes award recommendations to the Assistant Secretary for VETS. Selected VETS National Office management personnel and the Grant Officer discuss the grant review team recommendations and make final grant award decisions.

Evidence: 1.) Solicitations for Grant Applications (SGAs) posted on-line at www.fedgrants.gov and printed in the Federal Register. NOTE: In PY 2005 two grants were awarded sole source to a grantee who operates within the Katrina and Rita Disaster area. VETS anticipates awarding an additional $99K for continued assistance related to circumstances that arose from those disasters in PY 2006.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: VETS uses two primary methods for overseeing grant performance: quarterly desk reviews and on-site reviews. A desk review is a review of data in the VOPAR system and of reports submitted by the grantee. Grantees are required to submit the following reports: Quarterly Financial Reports; Quarterly Program Reports; a 90-Day Follow-Up Report (Final Report); and a 180-Day Follow-Up Report (Longitudinal Survey). Grantees are required to report their financial and programmatic performance throughout the grant period and for at least six months beyond the actual period of performance. All competitively-awarded grantees have 30 days after the end of each Federal fiscal quarter to submit their financial and technical performance reports into VOPAR. While VETS' desk reviews are the primary means of monitoring grantee performance and financial status, VETS staff go on-site to observe operations and review records to ensure that grantees also are meeting their obligations to track the on-going progress of individual program participants through a detailed, comprehensive case file record management system and procedures. Annual on-site monitoring visits are typically conducted late in the second quarter of the grant period and involve an in-person visit to the grantee by the GOTR, observation of grantee activities and interviews with grantee staff.

Evidence: Solicitations for Grant Applications (SGAs) posted on-line at www.fedgrants.gov and printed in the Federal Register. 2.) VETS' HVRP On-site review guide

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: HVRP performance data are reported annually in the USDOL Annual Report to Congress. In addition, the most recent full year data (PY 2004) for each grantee for is available on the VETS website. A summary chart related to all of the performance measures, goals, and actual results also is available on-line for public review.

Evidence: 1. Annual Report to Congress FY2005. 2.) VOPAR "Roll-Up" Reports for PY 2004.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: HVRP has two long-term performance measures: Entered Employment Rate and Employment Retention Rate. For both of these measures, VETS has increased HVRP's long-term performance goals each year from PY 2003 to PY 2011. Over the last three program years, PY 2003 through PY 2005, VETS has met or exceeded its ambitious targets for its Entered Employment Rate measure. For example, in PY 2003, the Entered Employment Rate exceeded the target by 6.5 percentage points. In PY 2004, the Entered Employment Rate result exceeded its target by 5 percentage points. In PY 2005, the Entered Employment Rate actual result for three quarters have met the established target. In PY 2005, the first program year after the Employment Retention Rate baseline measure was established, preliminary 3rd quarter results met the plan. VETS also expects PY 2005 full year results to exceed the established year-end targets for both measures. Based on these actual results and the limited experience with the Employment Retention measure , HVRP's performance merits a "Large Extent."

Evidence: 1. FY 2005 DOL Performance and Accountability Report.; 2. PY 2005 VETS Operations and Programs Activity Report (VOPAR) Quarterly and Final Program Year Reports, including 3rd quarter PY 2005 preliminary data to determine current Employment Retention Rate; 3.) Solicitations for Grant Applications (SGAs) posted on-line at www.fedgrants.gov and printed in the Federal Register. 4.) USDOL Strategic Plan for Fiscal Years 2007 - 2011. http://www.dol.gov/_sec/stratplan/strat_plan_2003-2008.htm. *NOTE: For Employment Retention Rate, the baseline target was established in PY 2004, consequently, VETS has actually increased that measure's target in each year from PY 2005 to PY 2011.

Does the program (including program partners) achieve its annual performance goals?

Explanation: HVRP has the same annual and long-term performance goals for its two annual performance measures: Entered Employment Rate and Employment Retention Rate. Since the annual and long-term performance measures are the same, meeting each annual performance target ensures that the program meets its long-term performance goals. Over the last three years, PY 2003 through PY 2005, VETS has met or exceeded its annual performance goals within each of its annual measures. VETS believes that the significantly large increase in the PY 2004 actual Entered Employment Rate may have been attributable to improved accuracy of the data resulting from the transition from largely a manual data collection system to the automated VOPAR system. Taking into account the HVRP's ability to meet its goals but the short experience with the Entered Employment Rate measure, its performance merits a 'Large Extent."

Evidence: 1. VETS Operations and Programs Activity Report (VOPAR) Quarterly and Final Program Year Reports; 2.) Solicitations for Grant Applications (SGAs) posted on-line at www.fedgrants.gov and printed in the Federal Register. *NOTE: For Employment Retention Rate, the baseline target was established in PY 2004, consequently, the HVRP actual result exceeded plan in the 3rd quarter of PY 2005 which are the latest data available.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The HVRP program has implemented numerous procedures to measure and achieve improved efficiencies and cost effectiveness. The primary efficiency measure is Average Cost per Placement, however, the program also tracks Average Cost per Participant and Average Wage at Placement to monitor program efficiency. HVRP has demonstrated consistent or improved efficiencies and cost effectiveness over the long term within all of its efficiency measures. Furthermore, when compared to increases in actual program appropriations, both the annual and long-term performance measures improvements exceed the overall rise in program expenditures. In reviewing past years' actual performance, the Average Cost per Placement decreased in PY 2004 and PY 2005*. Average Cost per Placement decreased in PY 2004 to $2,152 from $2,214 in PY 2003. Preliminary data in PY 2005 also indicates that the Average Cost per Placement is decreasing as compared to PY 2004 actual results. VETS believes that this improvement results from the increased level of grantee experience in managing their HVRP grants. VETS staff also instituted changes in the timing of on-site monitoring visits, the annual post award conference training, and the dissemination of standard operating procedures and definitions. These strategies have had the desired result of reducing the cost disparities across grantees, which has improved the program's overall effectiveness and efficiency. Based on the positive and consistent efficiency improvements, HVRP merits a "Yes."

Evidence: 1. VETS Operations and Programs Activity Report (VOPAR) Quarterly and Final Program Year Reports; 2.) Solicitations for Grant Applications (SGAs) posted on-line at www.fedgrants.gov and printed in the Federal Register. 3.) FY 2005 Performance and Accountability Report. http://www.dol.gov/_sec/media/reports/annual2005. *NOTE: PY 2005 Average Cost per Placement 3rd Otr. results are $2,102 which represents a continuous improvement from PY 2004 - the actual complete PY 2005 results also are expected to meet or exceed plan by the end of the program year.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Based on the experience of other programs that serve the homeless, including homeless veterans, HVRP's focus on employment and training services has identified a key factor in ensuring a successful end to homelessness. Representatives from veteran service organizations, the VA, and the National Homeless Veteran Coalition have recognized the HVRP as a critical program to address the needs of homeless veterans. Other homeless programs concentrate on providing more immediate needs such as emergency shelter, food, and substance abuse treatment. While these are critical components of any homeless program and HVRP grantees are required to demonstrate that their clients' needs in those areas are met, the objective of HVRP is to enable homeless veterans to secure and keep jobs that will allow them to re-enter mainstream society as productive citizens. One example of a Federal homeless program using the shelter/housing approach to reducing homelessness is the Department of Housing and Urban Development's (HUD) Homeless Assistance Grants program. The Homeless Assistance Grants program also tracks the Entered Employment Rate of its participants, which in 2005 was 17%. Comparing the 17% figure to HVRP's projected 61% Entered Employment Rate for PY 2005 reveals the soundness of HVRP's employment-based approach to increasing self-sufficiency and reducing homelessness. When compared to other VETS and Workforce Investment Act (WIA) employment and training programs, HVRP's outcomes are comparable, even though its participants face substantial barriers to employment. When compared to other targeted employment and training programs, such as the WIA Adult Program, WIA Dislocated Worker Program, and the Employment Opportunities for Youth and Adults with Disabilities Program, the HVRP Average Cost per Placement is actually less than those programs Average Cost Per Participant. In PY 2004, HVRP's Average Cost per Placement ($2,229.55) was actually less than both the Average Cost per Participant under the Dislocated Worker program ($3,318) and the Employment Opportunities for Youth and Adults with Disabilities ($2,882) during the same year. HVRP also compares favorably to the VA's Compensated Work-Therapy (CWT) and Compensated Work-Therapy/Transitional Residence Program. Through this program the VA offers structured work opportunities and supervised therapeutic housing for at-risk and homeless veterans with physical, psychiatric and substance abuse disorders. At discharge, 46 percent of the veterans were placed in competitive employment and 8 percent were placed in training programs in PY 2002 which is significantly less than the HVRP Entered Employment Rate of 60.3 percent during the same program year.

Evidence: FY 2005 Performance and Accountability Report. http://www.dol.gov/_sec/media/reports/annual2005/ ExpectMore.Gov Summary and Details of the HUD Homeless Assistance Grants program. http://www.whitehouse.gov/omb/expectmore/summary.10001234.2005.html Department of Veteran Affairs Fact Sheet, March 2002: VA Press Release and Fact Sheet: http://www.va.gov/pressrel/hmlssfs.htm

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: VETS has made a commitment to implementing regularly scheduled, objective, high quality, independent evaluations that assess how well HVRP is accomplishing its mission and meeting its long-term goals. VETS hopes to initiate a formal independent evaluation of the program in the next year or so. GAO has looked at certain aspects of the HVRP program at the request of the Subcommittee on Economic Opportunity, Committee on Veterans' Affairs, House of Representatives. GAO looked at DOL's expenditures on HVRP Grants and at the performance measures and goals to assess the effectiveness of HVRP.GAO also interviewed DOL program officials. GAO recommended that VETS adopt an employment goal for HVRP, which VETS has now implemented. VETS believes that the new retention goal will give HVRP an additional measure to ensure that it is effective and achieving the annual and long-term results that are intended.

Evidence: In addition, over the years, a number of individuals have testified before Congress about the effectiveness of the HVRP in an October 7, 1999 letter to colleagues, the Honorable Lane Evans, ranking Democratic member, Subcommittee on Benefits, stated, "The Homeless Veterans Reintegration Program -- a modest, cost-effective program designed to help homeless veterans re-enter and succeed in the job market -- has proven its worth since 1987?? Few government programs achieve so much for so little." In 2004, official from AMVETS Testified to Congress that, they, "support reauthorization of HVRP. It has been a very successful program, broadly supported by local community groups and veterans service organizations." In May 2005, Carl Blake's, Associated Legislative Director, Paralyzed Veterans of America, stated before the House Subcommittee on Veterans' Affairs, Subcommittee on Economic Opportunity, concerning the reauthorization of HVRP , that, "The Homeless Veteran Reintegration Program (HVRP) managed by DOL's Veterans Employment and Training Service (VETS) is a valuable program focusing on employment of homeless veterans. This program has achieved wonderful success since its inception almost 20 years ago. The HVRP provides help for those veterans with the most significant problems from substance abuse, severe PTSD, serious social problems, legal issues and HIV. The specialized services needed for these veterans are often their only hope." GAO: Testimony Before the Subcommittee on Economic Opportunity, Committee on Veterans' Affairs, House of Representatives, Homeless Veterans: Job Retention Goal Under Development for DOL's Homeless Veterans' Reintegration Program (HVRP). AMVETS Testimony. http://www.amvets.org/HTML/what_we_do/testimony_050504.htm. PVA HVRP Testimony http://www.pva.org/caphill/leg/testimony/2005/05004.htm