Detailed Information on the
Student Aid Administration Assessment

Improve contract oversight and performance management

Maintain progress on system integration efforts

Utilize developed unit-cost framework to make budgetary and management decisions

Improve program oversight and compliance

Implement a new data strategy that yields more timely and accurate financial and program data

Better integrate data into decision-making, including the development of a more comprehensive approach to eliminating program fraud and error

Complete development of a unit-cost framework and meaningful efficiency targets

Measure: Customer service level on the American Customer Satisfaction Index for the Free Application for Federal Student Aid (FAFSA) on the Web.

Explanation:The American Customer Satisfaction Index is an economic indicator that measures the satisfaction of consumers across the U.S. economy. The ACSI interviews about 80,000 Americans annually. Respondents are screened to cover a wide range of business-to-consumer products and services, including durable goods, services, non-durable goods, local government services, and Federal government agencies. ACSI data are used by academic researchers, corporations, government agencies, market analysts, investors, industry trade associations, and consumers. Improving service is one of the primary objectives of the FSA. This metrics represents progress towards achieving that objective. FAFSA on the Web is one of the program's most public/customer-facing products.

Measure: Direct administrative unit costs for origination and disbursement of student aid. This measure represents the administrative cost to originate and disburse a student loan or Pell Grant. The unit of measure to calculate the per-unit cost is the number of program disbursements. The total cost for origination or disbursement of student loans is thus divided by the total number of Pell and Direct Loan disbursements to arrive at these unit costs. The total number of disbursements is generated directly from the Common Origination and Disbursement system. Reducing costs is one of the primary objectives of the program. This metric represents progress toward achieving that objective.

Explanation:Unit costs are derived from the Department's Activity-Based Management program using direct administrative costs. They do not include administrative overhead or investment/development costs. Administrative overhead and development costs are not included, as the direct administrative cost is a more accurate measure of the operational costs of the process. Any operational improvements are more clearly identified in the direct costs, as only the costs or functions directly within the process owner's control are measured

Measure: Pell Grant improper payments rate. Improving program integrity is one of the primary objectives of the department. This metric represents progress towards that objective. Ensuring that recipients are not fraudulently awarded aid is the first-step in guarding the taxpayers' interests.

Explanation:This measure is estimated from records from Federal Student Aid's applicant file compared against statistical averages from the IRS. The improper payment rate has two parts (over- and under-awards), which are added together to estimate the overall rate. The rate represents the percentage of all payments made to Pell grant recipients that are not accurate.

Measure: Direct Loan recovery rate. Improving program integrity is one of the primary objectives of the FSA. This metric represents progress towards that objective. Minimizing defaults is the first step, but if the borrower fails to keep his obligation to repay the loan, recovering amounts owed will ensure that the taxpayers' burden is minimized.

Explanation:The recovery rate equals the sum of collections on defaulted federal direct student loans divided by the outstanding default portfolio of direct loans at the end of the previous year -- in other words, the percentage of outstanding direct loans in default status that are collected. This measure indicates the program's efficiency and effectiveness in program compliance and in protecting federal assets. It demonstrates how well the program makes sure that students repay their student loans even after they go into default status.

Measure: FFEL recovery rate. Improving program integrity is one of the primary objectives of the FSA. This metric represents progress towards that objective. Minimizing defaults is of course is the first step, but if the borrower fails to keep his obligation to repay the loan, recovering amounts owed will ensure that the taxpayers' burden is minimized.

Explanation:The recovery rate equals the sum of collections on defaulted Federal Family Education loans (FFEL) divided by the outstanding default portfolio of these same type of loans at the end of the previous year -- in other words, the percentage of outstanding FFEL loans in default status that are collected. This measure indicates the program's efficiency and effectiveness in program compliance. It demonstrates how well the program makes sure that students repay their student loans even after they go into default status.

Is the program purpose clear?

Explanation: The Higher Education Act provides six explicit purposes for the Office of Federal Student Aid (FSA): 1) to improve service to student aid program participants; 2) to reduce the cost of student aid administration; 3) to increase accountability for program management officials; 4) to increase student aid management flexibility; 5) to integrate student aid information systems; and 6) to better ensure student aid program integrity.

Evidence: The program's purpose is established in Section 141 of the Higher Education Act of 1965, as amended (20 U.S.C. 1018).

Does the program address a specific and existing problem, interest, or need?

Explanation: The program addresses the need to effectively administer $80+ billion in Federal financial aid for higher education. Each year, the federal government makes available more than $80 billion in grants, loans, and work-study to help students and parents pay for postsecondary education

Evidence: The Higher Education Act of 1965, as amended, authorizes several student financial aid programs, which are administered by the Office of Federal Student Aid

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program's purpose is to administer the federal responsibilities associated with the Department's student aid programs. While other state, local and private entities are involved in administering some aspect of these programs (particularly with respect to federal student loan origination, servicing, and default collection activities); they play no explicit role in federal activities.

Evidence: Section 141 of the Higher Education Act of 1965, as amended (20 U.S.C. 1018) established the Office of Federal Student Aid as a performance-based organization responsible for managing the operational functions supporting the programs authorized under Title IV of the Act. Student Aid Administration is the sole source of salaries and expenses funding for FSA in support of this responsibility.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that an alternative approach or mechanism would be more efficient in administering the Department's student aid programs. There have been key programmatic concerns raised by the Department's Office of the Inspector General (OIG) and the Government Accountability Office (GAO) reports, such as FSA's relationship with its non-federal "partners" (e.g., guaranty agencies); its diligence in its contract monitoring process; and shortcomings in detecting lender inducements. Despite these concerns, neither OIG nor GAO recommended major program design changes in their reports. Moreover, GAO removed the student aid programs from its high-risk list in 2005.

Evidence: GAO-05-207 Report, "High Risk Series," ED-OIG/A04E0009 Audit Report, "Review of Financial Partners' Monitoring and Oversight of Guaranty Agencies, Lenders, and Servicers," and GAO-07-750 Report, "Increased Department of Education Oversight of Lender and School Activities Needed to Help Ensure Program Compliance.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Student Aid Administration (SAA) is the salaries and expenses funding source for the delivery, administration and oversight of the Department's student financial aid. The primary office responsible for these activities is the Office of Federal Student Aid, which was established as a Performance Based Organization (PBO). The mechanisms funded by SAA for delivery, administration and oversight of postsecondary aid are the means by which the Department ensures that students eligible for aid receive that aid and those who are not eligible are not given financial assistance.

Evidence: Funded by SAA, the Department's Free Application for Federal Student Aid (FAFSA) is one of the primary means by which a student would apply for postsecondary financial aid. The mechanism, and the systems it feeds into - similarly support by SAA - are the primary determinant to ensure that resources address the program's purpose and reach the intended beneficiaries. Operating plans from the Office of Federal Student Aid, status of funds reports providing obligations by accounting codes and budget object classes, and budgetary justifications to Congress all provide additional evidence that resources provided by SAA are targeted to and reach the program's intended beneficiaries - the nation's postsecondary students.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Goal 3, Objective 2 of the Department's Strategic Plan is to deliver federal student aid to students and parents effectively and efficiently. In support of this Objective, FSA established five specific long-term measures focusing on tangible outcomes (these metrics represent a subset of about 20 metrics used to measure and monitor the success of the FSA and are based upon the objectives of the PBO enabling legislation.): 1) Direct unit costs for origination and disbursement of student aid, which measures the degree to which student aid is delivered in a cost effective manner; 2) Customer service level on the American Consumer Satisfaction Index for the Free Application for Federal Student Aid (FAFSA) on the Web, which measures the degree to which applicants for student aid are satisfied with the application process; 3) Pell Grant Improper Payments Rate, which measures how well the program is making certain that Pell grants are being made to only the proper individuals; and 4) Direct Loan recovery rate and 5) FFEL recovery rate, both of which measure how successful the program is in collecting outstanding loans in the two loan programs.

Evidence: In the 2007 Fall Update, ED revised its long-term performance measures making them consistent the Department of Education Strategic Plan. The Fall 2007 revision added one new measure for a total of the five listed in the explanation section. These measures are also published in Department's Strategic Plan and in the Justification of Appropriation Estimates to the Congress.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The targets and timeframes for each of the five long-term measures are dynamic and seek on-going long-term improvement with each consecutive year. Unit costs for delivery of student aid, levels of customer service, Pell Grant improper payment rate and recovery rates for Direct Loans and FFEL loans are all quantifiable, verifiable and have targets in excess of five years. In the area of customer satisfaction, FSA's target of 82 in 2007 on the American Customer Satisfaction Index exceeded the 72.9 score of the average governmental e-gov site for that same year.

Evidence: We do not have measures from the private sector - for Sallie Mae, for example-- against which to compare. Absent this, however, we can examine and compare trends in FSA's own measures and actual outcomes over the past several years in an effort to determine the degree to which the measures are ambitious. These long-term measures are published in the Department's Strategic Plan and in the Department's most recent FY 2009 Performance Budget.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The five long-term measures -- Direct administrative unit costs for origination and disbursement of student aid, Customer service level for FAFSA on the Web, Pell grant improper payment rate, Direct Loan recovery rate and FFEL recovery rate -- all measure annual progress toward SAA's long-term goals. These measures are quantifiable and outcome oriented. They support the Department's strategic plan and are consistent with FSA's strategic goals.

Evidence: These performance measures are included in the Department's Strategic Plan; The "Program Performance Measures" section of this document indicates the dual terms for these measures.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The targets and timeframes for each of the five annual measures are dynamic and seek on-going long-term improvement with each consecutive year. Unit costs for delivery of student aid, levels of customer service, Pell Grant improper payment rate and recovery rates for Direct Loans and FFEL loans are all quantifiable and verifiable. Each target aims at incremental improvement beyond the 2006 baseline. Beginning with a baseline of 80 on the Customer Service Satisfaction Index in 2006, the customer service goal aims for 82 and 83 in 2007 and 2008, respectively. Similarly, direct administrative costs for origination and disbursement of student aid begins in 2006 with a $4.24 per unit baseline and aims for $4.15 per unit in 2008 and $4.00 in 2009. Improper payments rates for Pell grants and recovery rates for direct loans and FFEL loans show a similar trend, beginning with a 2006 baseline and seeking incremental reductions for Pell improper payments and incremental increases in subsequent years for recovery rates in the two loan programs.

Evidence: We do not have measures from the private sector - for Sallie Mae, for example-- against which to compare. Absent this, however, we can examine and compare trends in FSA's own measures and actual outcomes over the past several years in an effort to determine the degree to which the measures are ambitious. In the area of customer satisfaction, FAFSA on the Web compares favorably to other federal, state and private sector entities. FSA's target of 82 in 2007 on the American Customer Satisfaction Index exceeded the 72.9 score of the average governmental e-gov site for that same year. These measures are published in the Department's Strategic Plan and in the Department's most recent FY 2009 Performance Budget.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The majority of major FSA contracts include performance incentives tied to service level agreement (SLAs). There needs to be, however, a stonger link between these agreements and the annual or long-term goals of the program.

Evidence: All of FSA's major contracts - with the exception of the Common Origination and Disbursement (COD) contract -- has incentives or disincentives written into them.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Government Accountability Office and the Department's Inspector General have conducted a number of audits and quality evaluations of FSA's processes and program effectiveness. These audits have been timely in their response to programmatic issues and have conducted on a regular basis. They have not been, however, of sufficient scope or depth in their look at FSA's business processes to support program improvements.

Evidence: The Office of the Inspector General has conducted the following audit reports: Increased Department of Education Oversight of Lender and School Activities Needed to Help Ensure Program Compliance - June 2007, Special Allowance Payments to the Pennsylvania Higher Education Assistance Agency for Loans Funded by Tax-Exempt Obligations - November 2007, Special Allowance Payments to Nelnet for Loans Funded by Tax-Exempt Obligations -- November 2007; Controls Over Contract Monitoring for Federal Student Aid Contracts - August 2007; and Review of Financial Partners' Monitoring and Oversight of Guaranty Agencies, Lenders, and Servicers -- September 2006.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Beginning with the recent FY 2009 Justification of Appropriation Estimates to the Congress ("Congressional Budget Justification"), the Department, in conjunction with the Office of Management and Budget, reconfigured its Student Aid Administration budget request in a manner built around the program's strategic goals. The annual request for funding is now tied to FSA Strategic Planning goals and makes clear the specific objectives supported while pointing out the consequences of failure to fund the objectives. In this way, the Strategic Plan links the level of resources requested with the success of the intended outcomes they will support. The Student Aid Administration budget includes all costs necessary, including both direct and indirect costs.

Evidence: The Department of Education Fiscal Year 2009 Justifications of Appropriation Estimates to the Congress ties together resources needed and performance goals, both annual and long-term. FSA's activity based cost model also shows the connection between SAA's budgetary investments and performance goals. The strategic goals are: Integrate FSA systems and provide new technology solutions; Improve program integrity to facilitate access to postsecondary education, while reducing vulnerability to fraud, waste, abuse and mismanagement; Reduce program administration costs; Improve human capital management; and Improve products and services to provide better customer service.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: In the Fall 2007 PART update, the Department revised Student Aid Administration's performance measures, adding one, amending four and deleting three. The effect of this effort was to ensure that FSA's goals were clear, specific, ambitious and integrated with the Departments' overall goals.

Evidence: FSA's 2007 Organizational Assessment (OA) is a series of about 20, key metrics or performance measures that are developed and monitored by ED Senior Leaders to evaluate FSA's annual performance. The metrics focus on overall Management, Human Capital, and Strategic Results. OA metrics are incorporated in the FSA Annual Performance Plan. FSA's revised acquisition strategy is the office's response to problems associated with an attempt to procure multiple business capabilities with single vendors as a part of the larger effort to replace multiple legacy systems. The revised strategy of multi-vendor contract vehicles reduced the risk of a single point of failure and increased competition among vendors.

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: Alternative analyses are included in business cases developed for all major investments in consultation with the Department's Office of the Chief Information Officer. Recent experience indicates some deficiencies in these analyses. For example, the ADvance Development risk analysis identified major schedule and feasibility risks; however, it is not apparent that these were used to guide a meaningful consideration of alternatives. This project has recently been cancelled. FSA's Activity Based Costing (ABC) model is used to drive down overall cost of significant processing activities, such as application processing and common origination and disbursement. Discussion of alternative approaches may result from these data; however, it is not currently used in the business case analyses, because most of the business cases focus on overhead types of investments, which do not necessarily have specific outputs or workload.

Evidence: Evidence of these processes include OMB Exhibit 300 guidance, annual FSA CPIC Working Group Sessions, Exhibit 300's and supporting business cases for FSA activities.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: FSA regularly collects information from its key partners and contractors, but the nature of the data collected and the linkage between these data and improved performance in important areas is not demonstrated.

Evidence: There are no data collected in the area of loan portfolio performance, a likely area in which performance data provided from a key program partner could improve performance. Likewise, the struggles FSA has faced in implementing its Data Strategy has compromised efforts in this area.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: All managers in Federal Student Aid are required to have a performance plan, which ties their compensation to performance results. Performance standards in each manager's performance plan are required to align with applicable action items in the Federal Student Aid Annual Performance Plan. Each of those action items in turns aligns with Federal Student Aid's top priorities and long-term goals.In addition, the Common Services for Borrowers (CSB) Contract, which handles Direct Loan Servicing, Debt Collection, and Direct Loan Consolidation, currently has 45 Performance Metrics, which are used to tie disincentives/incentives to performance. Punitive measures in the ADvance contract meant that after the Department terminated the development portion of the contract, the contractor absorbed the costs of further reductions in the legacy operations that the contract was written to replace. Similar sanctions were imposed on the CSB vendor. The ADvance vendor also was required to surrender the software code developed under the contract. Debt Collection contractors are paid, and receive more accounts, based on their performance. The top collection agency will receive the highest percentage of new accounts in each of the transfers and any collection agency that falls below a score of 55 will have accounts removed from their transfer percentage. The accounts removed from the lowest three collection agencies are given to the top three collection agencies.

Evidence: Internal managers' accountability is solidified through individual performance plans with specific standards, which are used for the annual performance appraisal process. These individual performance plans have explicit linkages to the FSA Annual Performance Plan. Performance ratings are documented on the performance plan and a copy is retained in Federal Student Aid's Human Resources office. The FSA performance plan identifies managers who are responsible for achieving key results, both in terms of key metrics and major project and milestone accomplishments. Key FSA contracts contain Service Level Agreements that provide key performance metrics to help achieve FSA program goals and tie contractor compensation to performance.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Spending plans on contracts in FSA are submitted in advance to the Office of the Secretary and through the Office of Contract and Acquisition Management. In this way, contract spending in FSA is regulated by an approved Acquisition Plan and any changes to this spending plan must be approved by an addendum request. Spending on large ticket information technology projects face further review by the Department's Investment Review Board, which is tasked with ensuring that information technology investments are consistent with strategic plans and the Department's mission. FSA also establishes and maintains operating plans for the funds allocated to it from the Student Aid Administration account. These operating plans provide information on time period (by month), and purpose (by object class). They indicate the activities supported by the funds and the schedule of spending. The same Oracle system on which the operating plans reside provides a plan verses actual report, squaring actual obligation and expenditure of funds against planned levels. This report shows that funds are obligated in a manner consistent with the operating plan, both in terms of the activities and the timing. Other reports compare allotted funds against actual obligations and expenditures by limitation and activity accounting codes.

Evidence: Unobligated funds at year-end, as determined by the SF 133, have not been in excess of .05 percent of the total funds made available annually for any year since the inception of the account. This information is taken from year-end SF 133, Report on Budget Execution and Budgetary Resources. Obligations have been made in a manner consistent with intended purpose. The largest category of spending in SAA -- information technology contracts -- is in fact controlled by the apportionment process in a manner that requires FSA to communicate to OMB any changes in the intended purpose for the funds.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: FSA has a number of IT improvements and other initiatives aimed at achieving efficiencies and cost effectiveness. Additionally, FSA maintains an Activity Based Costing (ABC) model, which is used to measure cost efficiency. This model provides the support and output for the program's efficiency measure: Direct administrative unit costs for origination and disbursement of student aid. This measure is aimed at the Department's Strategic Goal 3, Objective 2 - Deliver federal student aid to students and parents effectively and efficiently. Annual unit cost results are presented to Executive Leadership, and business unit contacts, with a full explanation of cost variances and a comparison to historical costs. Unit cost baselines and targets have been set in FSA's Five Year Plan. FSA is also revising the means by which major operational services are procured, with the goal of enhancing performance and driving down costs.

Evidence: The Pay.Gov solution created an interface between FSA FMS and the U.S. Treasury to support Automated Clearinghouse collections in the FFEL program. This solution improves data quality, eliminates data entry error and reduces manual intervention. FSA has also promoted electronic processes in the application and servicing areas resulting in cost efficiencies. FAFSA applications are just one example of this. The ABC model has shown a nearly 40% reduction in the unit cost of processing FAFSA applications since FY 2004, and a 28% reduction in the unit cost processing electronic FAFSA applications over the same period. In addition, the direct unit cost for the origination and disbursement of Pell Grants and Direct Loans has decreased by nearly 11% since FY 2004. Also, the direct unit cost for the Servicing of Direct Loan borrower portfolios has decreased 25% since FY 2004. Finally, the unit cost for the collection of defaulted loans has decreased 5% since FY 2004.

Does the program collaborate and coordinate effectively with related programs?

Explanation: FSA collaborates and coordinates effectively with many other federal agencies, including the Department of Veterans Affairs, the Social Security Administration, the Department of Homeland Security, the Selective Service Administration, the Department of Justice, the Department of Treasury (in particular, the Internal Revenue Service), the Department of Health and Human Services, and the Department of Energy.

Evidence: Applicant data is matched to various agencies to determine federal aid eligibility. For example, social security numbers (SSNs) are matched to the Social Security Administration databases to ensure a valid SSN was provided. A computer matching program with HHS's National Directory of New Hires assists FSA in the collection of defaulted student loans and grant overpayments. FSA is also working closely with the Internal Revenue Service to develop studies that estimate the amount and percentage of Pell Grant dollars erroneously awarded with the goal of improving internal processes to help identify error. In addition, to help improve the estimation of erroneous payments in the FFEL Program, FSA is collaborating with the Department of Energy's Oak Ridge National Laboratory to provide assistance in developing a conceptually valid and independent methodology, using available databases, to derive a procedure that will statistically assess and measure the risk of erroneous payments.

Does the program use strong financial management practices?

Explanation: The Department has been recognized in the past several years for taking steps to improve financial management and program integrity. FSA has particularly focused on remediating reportable conditions listed in annual audit reports. FSA established standard operating procedures to ensure that payments are made properly to minimize erroneous payments. These improvements and procedures included system modifications that support stronger program monitoring and oversight and compliance efforts. However, more work is needed on a methodology to address erroneous payments through the use of loan-level data.FSA was recognized for financial management improvements by removal from GAO's high-risk list in 2005. In addition, the program has received unqualified financial statement audit opinions since 2002. Some work remains to be done; for example, the FY 2007 Federal Student Aid Annual Report Ernst & Young's Report on Internal Control identified deficiencies in FSA's internal controls and recommended that FSA provide additional focus on program monitoring activities.

Evidence: Strong financial management practices are evidenced by: Removal from GAO's high-risk list in 2005 after 15 years on the list; Unqualified financial statement audit opinions received since 2002; No material weaknesses cited in the Financial Statement Audit Internal Control Report; Timely reporting was mentioned in 2003 but excluded thereafter; and Success in complying with accelerated financial reporting deadlines.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Office of the Inspector General (OIG) and the General Accountability Office (GAO) identifies management deficiencies by means of its audits. FSA has responded directly to these audits, particularly in the areas of lender oversight and in 9.5% special allowance payments for loans funded by tax-exempt obligations.

Evidence: In July 2007 the General Accountability Office (GAO) found fault with oversight tools in place to proactively detect potential instances of lenders providing improper inducements to schools in exchange for placement on a preferred lender list or other actions that limit the borrower's choice of lender. In addition to issuing new guidance on this issue, FSA has updated its program review procedures to include guidance that will help identify possible instances of improper inducements and limitations on borrower choice and to proactively identify potential program abuse. Additional evidence of FSA's response to this deficiency is seen in a 2008 Workforce Study aimed at realigning regional staff to improve FSA's oversight capabilities. FSA also consolidated all oversight functions in one area in May 2006 and thereafter began to write and implement similar national standards for lender and guarantor oversight. In response to an OIG finding of an organizational conflict of interest, FSA removed the State Agency Liaison was removed from the Financial Partners area. A number of FFEL lenders were exploiting a 9.5 percent floor for loans funded by tax-exempt obligations by using various means to expand the portfolio of loans eligible for the 9.5 percent special allowance rate. FSA determined retroactively the lenders exploiting this provision and published revised guidance, as well as imposing an audit requirement on the institutions.

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: Most of FSA's major contracts are performance based. FSA also conducts a rigorous planning and on-going evaluation process to ensure that the decisions it makes support the program's strategic objective, particularly where they relate to more efficient delivery of aid, improved customer service and greater oversight. Since 2004, FSA has been compliant with Education, OMB and FAR direction to require Earned Value Management on all development, modernization or enhancement of major IT acquisitions. FSA and its contractors have made great strides in employing EVM as a tool for measuring performance and understanding program risk. Projects have regular integrated baseline reviews and use the performance data to track variances and corrective actions.

Evidence: In business cases and in its Annual Performance Plan, FSA documents its plans and its activities. FSA also creates Project Management Scorecards and conducts baseline reviews for all significant undertakings. Most of FSA's major contracts collect performance data on a regular basis, and we use this data to provide monetary incentives and disincentives.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Performance goals for SAA identified as long-term are: Customer service level on the American Customer Satisfaction Index for the Free Application for Federal Student Aid on the Web, direct administrative unit costs for origination and disbursement of student aid, Pell Grant improper payments rate, and recovery rates for Direct Loans and FFEL loans. In FY 2007, FAFSA on the Web continued to score a customer satisfaction score of 80, a high score by ACSI standards. However, this score missed the performance target by a couple of points. FAFSA on the Web, which now handles aid applications for more than 90 percent of total applicants, faces continually challenging expectations from ever more Web-savvy customers. Direct administrative unit costs showed a trend in improvement and met its target in 2007. Federal Student Aid continues to improve financial management of Federal Student Aid contracts through integration of performance, budget, and acquisition processes. Goals for Direct Loan and FFEL recovery rates were met in fiscal years 2006 and 2007 and continue to improve. While customer satisfaction and the Pell grant improper payments rate came up short of their goals, the remaining long-term performance goals are on track to meet their targets.

Evidence: See the Program Performance Measures section of this PART document.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Customer satisfaction for FAFSA on the Web fell just short of the target in 2007, but unit costs for origination and disbursement of student loans exceeded the target for that year. Pell Grant improper payments rate also fell just short, but recovery rates for FFEL and Direct Loans exceed their 2007 targets. All five of SAA's long-term performance goals are also annual: Direct administrative unit costs for origination and disbursement, Customer service level on the American Customer Satisfaction Index for the Free Application for Federal Student Aid on the Web, Pell Grant improper payments rate, Direct Loan recovery rate and FFEL recovery rate. In the most recent year measured, the program achieved three out of five of its annual performance goals.

Evidence: See the Program Performance Measures section of this PART document.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Department has introduced significant new efficiencies in achieving student aid administration program goals. The performance measure for direct administrative unit costs for origination and disbursement of student aid has shown increasing efficiency and cost effectiveness in this core function. The ABC model has shown a nearly 40% reduction in the unit cost of processing FAFSA applications since FY 2004, and a 28% reduction in the unit cost processing electronic FAFSA applications over the same period. In addition, the direct unit cost for the origination and disbursement of Pell Grants and Direct Loans has decreased by nearly 11% since FY 2004. Also, the direct unit cost for the Servicing of Direct Loan borrower portfolios has decreased 25% since FY 2004. Finally, the fully loaded unit cost for the collection of defaulted loans has decreased 5% since FY 2004. FSA has been able to maintain and even improve service over recent years despite level and even decreasing appropriations. Some measure of efficiency was achieved through contract negotiation in that the cost of each successive unique recipient was at a lower cost.

Evidence: Among the purposes for which the PBO was created are to reduce the costs of administering student financial assistance programs and to improve service. The activity-based costing (ABC) model provides an objective assessment of the program's efficiency in the area of origination and disbursement of federal student aid. These areas of student aid delivery are one of the program's performance measures.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The program compares favorably in terms of customer satisfaction to other Federal, State and private sector entities. FAFSA on the Web received a score of 80, while the average governmental e-gov site received a score of 72.9 (ACSI, December 2007). All of FSA's scores together sum to a 79, but the average private sector score was a 76.3 and the average banking score was a 78 (ACSI, December 2007). On key metrics related to the program's loan portfolios, FSA conducts some comparisons between the Direct Loan Program and FFEL Program, namely delinquency rates and default rates. While the program compares favorably based on customer satisfaction indices, more avenues of comparison are needed to perform a complete analysis. For example, it would be helpful to compare unit cost data to private companies involved in similar activities.

Evidence: ACSI scores are provided in the 2007 PAR. Information regarding loan portfolio default rates and other metrics are available upon request.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The Government Accountability Office (GAO) and the Department's Office of the Inspector General (OIG) have provided numerous and in-depth evaluations of the program's operations. The record from these evaluations is mixed. They have pointed out weaknesses in the areas of oversight and program compliance and have identified issues with contract oversight. They have also highlighted improvements in program management. In 2005, GAO removed student financial aid programs from its High Risk list. FSA's CIO Office also conducts independent Post-Implementation Reviews (PIRs), which are designed to ensure compliance with IT management legislation and regulatory guidance.

Evidence: Evaluations from OIG have addressed the following areas: Increased Department of Education Oversight of Lender and School Activities Needed to Help Ensure Program Compliance, Special Allowance Payments to the Pennsylvania Higher Education Assistance Agency for Loans Funded by Tax-Exempt Obligations, Special Allowance Payments to Nelnet for Loans Funded by Tax-Exempt Obligations, Controls Over Contract Monitoring for Federal Student Aid Contracts, and Review of Financial Partners' Monitoring and Oversight of Guaranty Agencies, Lenders, and Servicers. GAO's High Risk series removed student financial aid programs from its list in 2005. The PIR evaluates IT investments in tem major categories and each category is rated based upon the extent to which the goals and objectives of each category are met.

Were program goals achieved within budgeted costs and established schedules?

Explanation: The program has recently experienced several major schedule variances, which FSA hopes to mitigate by revising its overall procurement strategy. While there have been variances in actual cost to the contractor from planned budget, these cost overruns do not affect the overall price of the contracts, which are Firm Fixed Price.

Evidence: Project Management Scorecards and regularly occurring baseline reviews provide documentation. One area of shortcoming has been the in the implementation of several major information technology projects that support the delivery of student financial aid -- ADvance (consolidating several legacy systems and providing a more seamless front-end interface) and Common Services for Borrowers. These two projects were delayed for various reasons, such as the complexity involved in replacing multiple legacy systems with single solutions and contracts, and requiring single vendors to provide competency in multiple business capabilities