Detailed Information on the
Housing Counseling Assessment

Establish information systems to collect client-level data from grantees, and help streamline and standardize the flow of information between HUD and its approved housing counseling agencies, and perform more in-depth data analysis at an aggregated level as a result of improved data election and reporting abilities.In FY 2007, the Housing Counseling Program piloted an initiative to collect client-level data from all agencies participating in the Program. HUD is mandating through federal regulation that all agencies participating in the Program use a client management system (CMS) that interfaces with HUD systems to provide this data to HUD starting October 1, 2007.

Commission HUD??s Office of Policy Development and Research to evaluate the program??s impact, performance, and ability to achieve established goals. In FY 2007, the Office of Policy Development and Research, began to implement a long term evaluation of the Housing Counseling Program. The research includes three components, a process evaluation, an outcome evaluation and an impact evaluation. In FY 2007 the evaluation team focused on implementation of the process evaluation.

Adopt standards for housing counseling programs and requirements for housing counselor credentials.

Establish efficiency measures to show improved administrative efficiencies and cost-effectiveness in achieving program goals.

Measure: Between FY 2004 and FY 2008, 3.5 million families will receive HUD housing counseling.

Explanation:This is a long-term output measure regarding the number of households receiving distinct housing counseling and/or group education services from agencies participating in HUD's Housing Counseling Program.

Measure: More than 80 percent of total mortgagors seeking help with resolving or preventing mortgage delinquency, and completed counseling, will successfully avoid foreclosure. (A6.2)

Explanation:This measure demonstrates the large percentage of households experience mortgage problems, that after completing default counseling, successfully avoid foreclosure.

Measure: The percentage of clients receiving pre-purchase counseling will purchase a home or become mortgage-ready within 90 days.

Explanation:This performance measure reflects both the degree to which pre-purchase counseling helps produce mortgage ready borrowers, and how it helps borrowers that need more preparation realistically assess their readiness for homeownership.

Measure: Minority clients are at least 50 percent of total clients receiving housing counseling in FY 2008 (A2.7)

Explanation:This measure incentivizes participating agencies and program staff to utilize innovative techniques to aggressively reach out to emerging markets and minority communities and make them aware of these critical services.

Measure: The percentage of One on One Counseling out of total counseling activity.

Explanation:

Measure: The percentage of clients receiving rental or homeless counseling either find suitable housing or receive social service assistance to improve their housing situation.

Explanation:This performance measure captures the positive impact of rental and homeless counseling.

Is the program purpose clear?

Explanation: The primary objectives of the program are to expand homeownership opportunities and improve access to affordable housing through the provision of a wide variety of housing counseling services to potential homebuyers, homeowners, low- to moderate-income renters, and the homeless. Counselors provide guidance and advice to help families and individuals improve their housing conditions and meet the responsibilities of tenancy and homeownership.

Evidence: Section 106(a)(2) of the Housing and Urban Development Act of 1968 (12 USC 1701x) provides the legislative authority for HUD to provide housing counseling services directly or through private or public organizations with special competence and knowledge in counseling low and moderate income families.

Does the program address a specific and existing problem, interest or need?

Explanation: The program educates potential and current homeowners on avoiding predatory lending practices included inflated appraisals, unreasonably high interest rates, unaffordable repayment terms, and other conditions that can result in a loss of equity, increased debt, default, and eventually foreclosure.

Evidence: In a 2002 study by Ohio State University, counseling was shown to reduce the incidence of defualt among low-income borrowers. In a 2001 Freddie Mac study, pre-purchase counseling was found to reduce 90 day delinquencies.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program does not duplicate other efforts, instead awards funds on a competitive basis to supplement the non-federal resources of existing local, regional, and national housing counseling organizations.

Evidence: HUD's housing counseling program is the only dedicated source of federal funding for housing counseling services.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program does not have major design flaws that limit its effectiveness. However, while the value of housing counseling is recognized, there is no agreed upon standard for this type of education.

Evidence: While the Ohio State University and Freddie Mac studies draw general conclusions on the effectiveness of different types of counseling techniques; mainly that one-on-one counseling is more effective than group sessions, HUD's housing counseling program allows less proven methods. For the past two years HUD's grant application scoring process has awarded higher points to applicants that provide a higher percentage of one on one counseling.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The program targets potential homeowners, current homeowners, low-income renters, and the homeless.

Evidence: Funds are awarded through the annual notice of funding availability (NOFA) based on the need served by the counseling agency. The housing counseling program guidebook defines eligible clients the counseling agencies can serve.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has one long-term outcome goal which measures the program's ability to counsel families delinquent on their mortgages to avoid foreclosure. The program also has two long-term output measures that set goals for the number of households counseled and the percentage of minorities served.

Evidence: HUD's 2003 ' 2008 Strategic Plan includes three long-term outcome and output performance measures.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has targets for its long-term measures. However, some long-term targets are at or just below current performance.

Evidence: The program's long-term goal that 65 percent of families receiving delinquency counseling will avoid foreclosure by 2008 has already been far exceed by actual experience. Currently, almost 92 percent of families receiving delinquency counseling successfully avoid foreclosure. Similarly, the program's goal that by 2008 at least 50 percent of its clients will be minority is nearly met today with 48 percent of clients being minority.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has three annual performance measures, which can be demonstrated as contributing toward the accomplishment of its long-term goals.

Evidence: The Annual Performance Plan (APP) includes three annual performance measures that are linked to its long-term goals. They are: H.1.7 'Housing Counseling is provided to 476,084 homebuyers in 2006'; H.2.5 'Housing Counseling is provided to 401,898 minority clients in 2006 '; and H.6.2 'More than 62 percent of total mortgagors receiving default counseling will successfully avoid foreclosure."

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has targets for its annual measures. However, some targets are at or just below current performance.

Evidence: The program's annual measure that 62 percent of families receiving delinquency counseling will avoid foreclosure has already been far exceeded by actual experience. Currently, almost 92 percent of families receiving delinquency counseling successfully avoid foreclosure. HUD's APP does not include baseline data for H.1.7 which makes it difficult to assess whether the program achieves its annual goals.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Grantees and sub-grantees commit to and work toward annual and long-term goals.

Evidence: All HUD-approved counseling agencies are required to report annual performance data via form HUD-9902 which is derived from program performance goals. Grantees contribute to the achievement of these goals by complying with commitments outlined in their funding application and subsequent reporting requirements based on the program's performance objectives.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: HUD closely follows independent research performed by academic institutions evaluating the effectiveness of housing and credit counseling or identifying best practices. However, there have been no evaluations in the form of studies, reports, or audits of HUD's housing counseling program.

Evidence: Program managers have requested that HUD's Office of Policy Development and Research include a Housing Counseling program evaluation on the 2005 research agenda.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget requests correspond to the annual and long-term goals for the number and type of households counseled and the percent of mortgagors receiving default counseling who successfully avoid foreclosure. Annual program goals also support the President's larger goal to expand minority homeowners by 5.5 million by 2010.

Evidence: Congressional budget justifications outline the number and type of households counseled with new funding requests. For example, in 2005, the request for $45 million ($5 million over 2004) HUD estimates will serve almost 60,000 additional households over the previous year's activity level.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Annually, HUD evaluates the program's performance against goals identified in the Strategic Plan and Annual Performance Plans. In response, program managers develop and implement measures to correct weaknesses.

Evidence: In 2002, to address data collection weaknesses, the program overhauled the primary data collection instrument and is now procuring a Client Management System to collect more meaningful client-level data.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Grantees are required to submit quarterly performance reports that include data on the number of clients served, staff hours, unit costs and, if applicable, allocations to affiliates and sub-grantees as well as a detailed accounting of the use of administrative funds. Actual activities and accomplishments are compared to projections submitted in the grant application to assure grantees are on target.

Evidence: All HUD-approved, HUD-funded housing counseling agencies file performance data electronically through form HUD-9902. All organizations that fail to report are removed from the HUD-approved list and are not eligible to apply for grant funds. For 2003, the program had an 82% response rate (1120 responses from 1320 agencies in database).

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: HUD withholds funding if grantees are out of compliance or have misused grant funds. In accordance with OMB Circulars, Grantees must maintain supporting documentation of all expenses for which they claim reimbursement. The Grant agreement stipulates that no payment requests shall be approved for a Grantee who has an overdue report until the report has been submitted by the Grantee and approved by HUD. HUD may also require reimbursement of grant funds that it determines were misused. HUD performs on-site monitoring of all grantees every two years and whenever remote monitoring or consumer complaints indicate a potential problem.

Evidence: The grant agreement and program guidebooks outline performance schedules, requirements, and sanctions. Approximately 5 percent of grantees have been removed for poor performance.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds are committed, obligated and expended in a timely manner for the intended purpose.

Evidence: Approximately two-thirds of all funds are obligated by the end of the second year after appropriations and nearly 100 percent is obligated by the end of the third year.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does not have a formal system in place to evaluate the efficiency or cost effectiveness of program execution. However, the program is working to procure a Client Management System that will enable program managers to improve oversight and program efficiency.

Evidence: HUD is in the process of procuring a Client Management System.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The housing counseling program works in conjunction with other state and federal homeownership programs, including mitigating risks in FHA's single-family mortgage insurance. Additional funding awards are made for proposals that serve HUD's public housing and Section 8 housing choice voucher homeownership programs.

Evidence: The 2004 NOFA will award additional funds for proposals that serve other HUD homeownership programs, such as public housing and housing vouchers. FHA's new zero downpayment mortgage insurance proposal would require borrowers to receive housing counseling as a condition of receiving insurance.

Does the program use strong financial management practices?

Explanation: The program has procedures in place to ensure payments are made properly for the intended purpose. All grants are paid on a cost-reimbursement basis. Grantees must maintain documentation for all costs and services provided as well as submit quarterly performance and financial reports to HUD. HUD temporarily suspends reimbursement to grantees that have not submitted required reports within thirty days of the quarter until the reports are submitted.

Evidence: There have been no material control weaknesses reported by auditors of this program.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program does not have a formal system in place to evaluate and improve management deficiencies. However, the program is working to procure a Client Management System that will enable program managers to improve oversight capabilities.

Evidence: HUD is in the process of procuring a Client Management System.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: HUD's housing counseling funds are awarded annually through a NOFA which ranks applications based on five rating factors: capacity, need, soundness of approach, leveraging resources, and achieving results and program evaluation.

Evidence: For the 2003 NOFA, HUD received 514 applications and ultimately awarded 442 grants. The average score of all applicants was 76 percent.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: HUD collects quarterly and annual data electronically and conducts on-site monitoring of all grantees every two years and whenever remote monitoring or consumer complaint indicatecomplaint indicates a potential problem.

Evidence: HUD reviewed approximately 283 of 807, or 34% of all approved counseling agencies and approximately 33% of grantees last year.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: HUD collects data on the local, regional, and national housing counseling agencies it funds. Data is not made public, but is available upon request.from all HUD-approved local, regional, and national housing counseling agencies. The data is published on the housing counseling pages of HUD's website.

Evidence: Grantee performance data collected through HUD form 9022 is published on-line at www.hud.gov/offices/hsg/sfh/hcc/03hcrprt.pdf

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: HUD's Strategic Plan establishes a goal that 3.5 million households will receive housing counseling between 2004 and 2008. HUD does not have performance data for that period yet nor are the annual targets ambitious enough to achieve the long-term goal for 2008. Targets for the program's other two performance goals are not ambitious given they exceed or are just below current performance.

Evidence: .For the program to achieve its long-term goal to serve 3.5 million households, it would need to serve 875,000 clients a year, but annual goals for 2005, for example, only aim to serve 476,084 clients. Another program goal, to serve at least 50 percent minorities is set at current performance and far below the 58 percent achieved in 2000. The goal that 65 percent of clients delinquent on their mortgage successfully avoid foreclosure is drastically lower than the last four years of experience which has hovered around 90 percent.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Performance reports submitted by grantees show HUD and its partners achieve and exceed some performance goals.

Evidence: HUD's 2003 Performance and Accountability Report shows the program planned to serve 238,356 homebuyers or owner in 2003 and actually served 257,096.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program does not have a way to demonstrate improved efficiencies or cost effectiveness in achieving program goals. However, the program is working to procure a Client Management System that will enable program managers to improve oversight and program efficiency.

Evidence: HUD is in the process of procuring a Client Management System.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Unlike the other government programs with similar purposes and goals, HUD's Housing Counseling Program is the only government program that makes the funding of individual counseling a priority over group education.

Evidence: The 2004 NOFA ranks higher applications that support individual counseling, in which the counselor examines and analyzes an individual or family's unique financial situation.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: There have been no evaluations in the form of studies, reports, or audits of HUD's housing counseling program.

Evidence: Program managers have requested that HUD's Office of Policy Development and Research include a Housing Counseling program evaluation on the 2005 research agenda