Detailed Information on the
Small Business Development Centers Assessment

Establish a nationwide SBDC counselor certification program to ensure quality technical assistance services are provided.

Reengineer the SBDC program examination process used by SBA to place more emphasis on performance analysis.

Incorporating a risk-based approach to the financial reviews to better link level of review with level of risk.

Enhancing the Baldrige-based accreditation standards to focus more on performance results.

Measure: Long-term clients counseled (#) (New measure, August 2007)

Explanation:

Measure: Cost Per Client Served

Explanation:The allocations for nascent entrepreneurs and exisiting business clients are estimates based on a weighted formula.

Measure: Small Businesses created (New measure, August 2007)

Explanation:The allocations for nascent entrepreneurs and exisiting business clients are estimates based on a weighted formula.

Measure: Percent customer satisfaction

Explanation:SBA completed the first year of its division-wide impact study for FY 2004 so FY 2004 represents a change from previous years.

Measure: Capital Infusion ($) (New measure, August 2007)

Explanation:SBA completed the first year of its division-wide impact study for FY 2004 so FY 2004 represents a change from previous years.

Measure: Cost Per Small Business Created (New measure, added February 2008)

Explanation:

Is the program purpose clear?

Explanation: The SBDC program provides business counseling and management assistance to current and prospective small business owners. It also assists in economic development such as trade shows, analysis of community's strengths and weaknesses, etc. The SBDC program is SBA's largest resource partnership, operated with state and local governments and institutions of higher education.

Evidence: 15 USC 648 and 13 CFR Part 130

Does the program address a specific and existing problem, interest or need?

Explanation: The problem is that 80 percent of new business starts discontinue operations within five years. Among the most prevalent reasons given for the discontinuance is lack of "knowledge." Also, there is a need to promote economic development by helping people with good ideas start and/or expand businesses. SBDCs also promote economic development by facilitating trade shows and conducting studies of the problems affecting businesses in a particular area and making efforts to remedy these problems. The program can address these problems and needs. Through resource materials, on-site counselors, and training seminars, the program provides access to business knowledge and increases economic activity.

Evidence: Dun & Bradstreet Annual Survey of Company Failures, SBDC Annual Reports, Chrisman Report

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: While a national program, SBDC is a public/private partnership designed to be state specific, and it provides long-term, in-depth counseling to small businesses. With 1,100 services centers nationwide, no other program covers such a wide geographical area. While the program has overlap with other technical assistance vehicles, e.g., Womens Business Centers, SBDCs administer unique programs like the Drug-Free Workplace, Compliance Alliance, & Defense Economic Transition. Additionally, SBA seeks to target the small business population that is not currently served by the private sector. Each SBDC conducts an annual needs assessment to determine its client-base.

Evidence: MIS Reports and Cooperative Agreement. There are other programs -- federal, state, local -- that assist small businesses. While they do provide some similar services, none are identical in scope or outreach to the SBDC Program.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: A major shift was made in the program design when it was determined that institutions of higher learning could more appropriately serve as state lead centers than State governments. This shift is considered a huge advantage for the program because of the depth of resources, administrative support, stability, and knowledge it provides. Also, the program leverages the Federal dollars by requiring a 100% match from State and local sources, adding to the program's effectiveness, outreach ability, and prevention of duplicative efforts. In an effort to make the program more competitive (currently a block grant), for the second year in a row, in its FY 2006 legislative package, SBA has proposed opening SBDC lead center grants to competitive bidding every 5 years. This would help increase efficiency in the program through promulgation of best practices.

Evidence: 15 USC 648; 13 CFR 130, FY 2003 Performance & Accountability Report, FY 2005 Budget and Performance Plan, reauthorization proposal.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The intended beneficiaries are nascent, start-up, & existing small businesses, a market of over 18 million people. With approximately 1,100 centers & 6,000 experienced counselors & consultants nationwide, the SBDC Program has the capacity to target this market. SBDCs are required to develop and conduct needs assessments to determine their client-base and target them, and to document whether program offerings are meeting local needs. Also, SBDCs have the authority and ability to contract for specialized services for clients not available elsewhere, thus increasing their ability to serve their clients. Their ability to draw on host institution resources (universities, colleges) and draw on resources of their economic development partners in the community increases their effectiveness. While the formula-based funding ensures that a proportionate number of citizens in each State have access to the Federal resources, this lack of competition (and subsequent promulgation of best practices) undermines the program purpose.

Evidence: MIS data, Chrisman Report, SBA FY 2005 Budget and Performance Plan. Economic impact data indicates that small businesses assisted by SBDCs in FY2001 had an additional $6.56 billion in sales and created/retained approximately 131,000 jobs.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The SBDC program has one long-term outcome-oriented performance measure: jobs created or retained. The remainder of the program's measures are output oriented. SBA is in the process of developing a long-term research plan to more accurately test and measure SBDC and other programs in terms of economic impact outcomes. As this project is developed, program goals and incentives will be reformed to address these measures.

Evidence: SBA Strategic Plan for FY 2003

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The SBDCs must create or retain 500,000 jobs by 2007. This is tracked annually. The SBDCs must increase counseling & training of established small businesses, rather than pre-ventures. This is tracked quarterly.

Evidence: MIS data and SBA Strategic Plan

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The SBDC program has established a limited number of annual goals to serve the needs of the small business community. Each SBDC has a needs assessment process in place as required by the Certification Standards. In addition, each SBDC is required to provide training and counseling to a specified number of clients based on assessed needs. These milestones/goals are part of the funding instrument, the co-operative agreement. The SBDCs also have a client satisfaction goal.

Evidence: MIS data; annual narrative reports; cooperative agreement; client satisfaction surveys; SBDC Chrisman Economic Impact Study

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The SBDCs have been given ambitious annual goals through the district offices. Although funding has increased only approximately 1% in nominal dollars, and has actually decreased in real dollars due to inflation, the SBDCs goals were increased by 3%. Under the existing measures, SBDCs created or retained 131,000 jobs in FY2001. SBDCs counseled and trained 295,000 established businesses in FY2002.

Evidence: MIS data and District Office Goals

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: SBDC partners support the overall annual performance goals of the agency and report their activities related to those goals. The SBDC Program Announcement requires all SBDC Lead Centers to ensure they receive all economic impact data from all service centers and that the data collected is consistent. The data required from all partners includes the long-term goal of jobs created and jobs retained.

Evidence: SBDC Program Announcement, MIS Data , Co-operative Agreement

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Office of Inspector General conducted a study of the SBDC Program and conducts studies of individual SBDCs. SBA's Office of Policy & Planning conducted a review of the SBDC Program. SBA's Office of Entrepreneurial Development will contract for a study during 2003. In addition the ASBDC has contracted for the Chrisman Economic Impact Study.

Evidence: OIG studies, "SBDCs: A Program Review, September 2001 Chrisman Study

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The FY 2004 budget request directly identifies quantifiable performance targets derived from a FY 2002 baseline and the resources necessary for their accomplishment. Despite a legislative requirement to allocate funds according to a population-based funding formula, states are required to meet their annual and long-term performance goals as stipulated in their Cooperative Agreements. SBA has proposed, as part of congressional reauthorization, to require each lead SBDC to compete in a competitive bidding process every five years and, as part of its Scorecard projects, is working with a task force to develop a legislative proposal to develop an incentive program for funding..

Evidence: MIS data and legislative initiatives; Cooperative Agreements; SBA Scorecard; FY 2004 Legislative Package

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Office of SBDCs' strategic plan contains the development of a needs assessment process which will identify long-term needs and, therefore, long-term goals for the program. In addition, individual SBDCs are required by certification standards to have their own strategic plans. Also, SBA is working with the SBDCs toward a broad SBDC strategic plan that contains measurable outcomes.

Evidence: Certification standards, OSBDC's and SBA's strategic plans. SBA, as mentioned above, is working with OMB officials, independent researchers, and members of academia to develop new and better economic impact measures and a more reliable testing model.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: SBA collects quarterly counseling & training statistics and annual economic impact data from the SBDCs. The SBDCs also prepare a semi-annual and annual narrative report documenting their progress against their goals. SBA uses this information to manage the program. Nonetheless, SBA has historically encountered some difficulty in aggregating data because some SBDCs do not use standard data definitions. When an SBDC is deemed to be not successful, action is taken. This includes conditional funding or termination.

Evidence: MIS data; SBDC Chrisman Economic Impact Study; narrative progress reports; OMB Circular A-110; 13 CFR 143

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and program partners are held accountable for cost schedules and performance results. Each year the SBA district offices and the SBA state directors negotiate milestones in the co-operative agreement. The failure to meet goals may lead to the termination or non-renewal of an SBDC. For example, the Louisiana SBDC and the Washington, DC SBDC were recently conditionally funded. They had to improve their achievements toward the goals in order to receive unconditional funding. SBA project officers also have a performance standard written into their PMAS, and their oversight of the SBDC program is reviewed during the Quality Service Review (QSR) process.

Evidence: See SBA Project officer PMAS , Co-operative Agreements, QSR reports, Letters to Louisiana and Washington DC SBDCs

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: In accordance with statute and agency practice, all funds are obligated on a fiscal year or calendar year basis in a timely manner; all funds are distributed in accordance with a statutorily required funding formula.

Evidence: 15 USC 648 and the JAAMs system.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Under SBA's activity based costing methodology, the federal cost of delivering services to SBDC clients is measured and tracked.

Evidence: SBA's activity based costing system

Does the program collaborate and coordinate effectively with related programs?

Explanation: As a public/private partnership, the SBDCs are experienced with collaboration and coordination. In fact, the SBDC program announcement requires SBDCs to work with other resources. They conduct joint training sessions with other SBA programs, such as SCORE and WBC and make mutual referrals to those resources. They also collaborate & coordinate with non-SBA programs such as MEP. While MEP is handling engineering questions for a small manufacturer, the SBDC can handle the business-related questions for that same manufacturer.

Evidence: SBDC program announcement and cooperative agreement

Does the program use strong financial management practices?

Explanation: The program adheres to the Small Business Act requirement that each SBDC be financially examined every two years. In addition, a year-end reconciliation is performed comparing the SBDCs financial reports to SBA's financial systems.

Evidence: See OSBDC financial reports and year end reconciliation.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Management deficiencies can be identified during financial and programmatic examinations. They are immediately addressed by SBDCs and SBA District offices. In addition, examiners follow-up on these deficiencies during their next biennial examination. If an SBDC does not address the deficiencies, they can and have been terminated or they can receive conditional funding or payments to the SBDC can be suspended. Deficiencies found during the certification process must also be addressed by the SBDC in order to receive funding. An SBDC can receive certification with conditions. In addition, SBA provides technical assistance to SBDCs that have deficiencies. For example financial training was provided to the California and Michigan SBDC Networks.

Evidence: Financial & Programmatic Review Reports, Certification Reviews Louisiana and Washington DC Letters

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: SBDCs receive bi-ennial programmatic and financial on-site examinations from SBA Headquarters. SBDCs also receive annual reviews by SBA District Offices. SBDCs are reviewed every four years by the Association of SBDCs in order to receive certification. SBDCs are reviewed annually by their host organizations. All evaluations are used to determine whether the SBDCs are accomplishing their mission.

Evidence: Financial and Programmatic review Reports, and SBDC Annual Narrative Reports

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: SBA collects substantial data from each SBDC network. This data is tabulated for each SBDC, for SBDCs by Region, and for the overall SBDC program. It is tabulated by quarter and by year. SBDCs also write annual narrative reports. SBDCs disseminate this data to Congress and to the public.

Evidence: MIS data, SBDC Annual Narrative Reports

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The agency has articulated its new long-term goals for this program in its Strategic Plan. Both annual performance goals and performance measures have been developed and are contained in both the Strategic Plan and the research plan for the economic impact study. Goals include creating or retaining jobs, increase counseling of established small businesses, and increased on-line counseling.

Evidence: SBA Strategic Plan

Does the program (including program partners) achieve its annual performance goals?

Explanation: The SBDC program met or exceeded its annual performance goals four out of the last five years. The SBDCs are required to input their counseling and training statistics in the MIS System which collects performance on numerous outputs and outcomes. They must also state their achievements toward their milestones (goals) in their annual narrative reports. In addition, the SBDCs must conduct client satisfaction surveys.

Evidence: MIS data, SBDC annual narrative reports, FY2001 Performance & Accountability Report and SBA Strategic Plan. In FY2002, SBDCs counseled or trained 651,421 small businesses; the goal was 631,349. Another performance goal was to develop a draft on-line needs assessment by September 2002; it was developed by February 2002.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The SBDCs have demonstrated improving efficiencies in achieving program counseling and training goals each year. Funding is fairly level, yet the SBDCs have increased counseling and training hours provided to small businesses. From 1999 to 2002, the unit cost of providing counseling to clients decreased from $170 to $154.

Evidence: MIS data, FY 2004 Budget request

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The SBDC Program provides long-term, in-depth counseling, averaging 5.5 hours of counseling to each client. Other than SBA's SCORE program, which is volunteer based and has fewer average hours of counseling, it is SBA's most cost efficient program per client served. The Chrisman Economic Impact Study shows that SBDC long-term counseling generated approximately $2.09 in tax revenues for every $1 spent on the entire SBDC program. In FY2001, SBDCs reported over 131,000 jobs created or retained, an increase in sales of $77 for every Federal dollar spent, $287M generated in State taxes, and $294M generated in Federal taxes. The program compares favorably to the NIST MEP Program which assisted 149,000 small manufacturers since 1989; SBDCs assist almost 700,000 small businesses every year. MEP's 2003 budget was $105.3M; the SBDCs 2003 budget was $88.4M. Since FY 2000, the cost per client for the SBDC program has steadily decreased from $162 to $135.

Evidence: MIS data, SBDC Chrisman Economic Impact Study, MEP web site, upcoming program evaluation.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The latest Chrisman Economic Impact Study shows that the SBDC program is effective and achieving results. It shows that long-term counseling activities generated approximately $2.09 in tax revenues for every $1 spent on the entire program (counseling less than 5 hours and training). It also shows that in FY2001 small businesses receiving counseling had an additional $6.56 billion in sales and created/retained approximately 131,000 jobs. A recent internal SBA independent study indicated that the SBDC program has numerous strengths. The long-term goals established this year will also help determine whether the program is achieving meaningful results. In addition, SBA has one contract in place to develop a periodic independent evaluation of the program and, in 2003, contracted with IBM to perform an program review of the SBDC program. SBA has also developed an annual impact study that was approved by OMB in 2004 to provide baseline economic data and annual output and outcome data to analyze performance. The initial study is being conducted in FY 2004.

Evidence: SBDC Chrisman Economic Impact Study (Jan 2003); and "SBDCs: A Program Review Sept 2001"