Detailed Information on the
Service Corps of Retired Executives Small Business Assistance Assessment

A SCORE Pilot Call Center is being established to provide a customized answering service for SCORE chapter telephones and will initially be able to register clients for local workshops using an online system. The call center is a response to leveraging volunteer hours and expertise by removing the routine administrative burden of telephone answering and workshop registration from the volunteers. The volunteers thus will focus more on the delivery of SCORE's core competency of business counseling and training.

Refining or improving performance measures to define audience reach.

Measure: SCORE Counseling - Number of clients counseled

Explanation:The FY2005 values that are indicated are estimates. During FY06 SCORE will implement the new client counseled definition. The number of clients counseled in FY 2006 will be the baseline for future counseling goals. Full implementation of the Entrepreneurial Development (ED) client definitions will provide predictability on which to achieve annual and long term objectives.

Measure: SCORE Online Users Counseled Online

Explanation:The FY2005 values that are indicated are estimates. During FY06 SCORE will implement the new client counseled definition. The number of clients counseled in FY 2006 will be the baseline for future counseling goals. Full implementation of the Entrepreneurial Development (ED) client definitions will provide predictability on which to achieve annual and long term objectives.

Measure: SCORE Training - Number of businesses trained

Explanation:The FY2005 values that are indicated are estimates. The number of clients trained in FY 2006 will be the baseline for future training goals. Full implementation of the Entrepreneurial Development (ED) client definitions will provide predictability on which to achieve annual and long term objectives.

Measure: % Customer Satisfaction

Explanation:SBA completed the first year of its division-wide impact study for FY 2004 so FY 2004 represents a change from previous years.

Measure: Small Businesses Created (New measure, August 2007)

Explanation:

Measure: Program cost per volunteer hour contributed

Explanation:By statute, provision of SCORE counseling, mentoring and training services is through a volunteer cadre of retired and working business professionals with expertise in owning, operating or managing a business.

Is the program purpose clear?

Explanation: SCORE, started in 1964, is SBA's oldest Entrepreneurial Development program. Staffed by volunteer working and retired executives and business owners, the program provides counseling and mentoring to entrepreneurs and existing business owners.

Evidence: SBA's Cooperative Agreement and the National SCORE Associations (NSA) Mission and Vision Statements

Does the program address a specific and existing problem, interest or need?

Explanation: 80 percent of new business starts discontinue operations within five years. Among the most prevalent reasons given for the discontinuance is lack of "knowledge." The program provides business advice through a network of volunteers and provides mentoring.

Evidence: Dun & Bradstreet Annual Survey of Company Failures

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program matches retired and active business executive volunteers with entrepreneurs for technical training and counseling. There is no identical duplication in federal government, although business training is available through other SBA programs. Additionally, there are some private sector programs (limited) that provide similar services.

Evidence: SBA OIG Inspection Report: "Coordination and Performance Measurement in SBA's Entrepreneurial Development Programs" September 2000, SBA budget request and Performance Plan, 2004

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program relies on 10,900 volunteers in 389 chapters who donate more than 1.1 million volunteer hours per year. These volunteers are supported by the National SCORE Association (NSA) staff of 13 to manage operations. The program is administered by one full-time SBA staff.

Evidence: SCORE Daily Desktop Guide; SCORE Operating Manual; SCORE Training Manual; SCORE Marketing Guide; SCORE Fundraising Guide; Cooperative Agreement

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: SCORE is a unique program, manned by a counseling team of over 10,500 volunteer Executives -- experienced business leaders from every industry. SCORE has aggressively reached beyond the walls of its chapters to reach entrepreneurs. SCORE actively seeks and forms alliances with leading companies to expand small business outreach, services and resources to communities nationwide -- companies like Lowe's Home Improvement, for example, where an alliance is developing to provide counseling to Lowe's largely entrepreneurial clientele nationwide, in their store locations. Online counseling is now over 33% of SCORE business. The SCORE program serves primarily the nascent entrepreneur market (63% of clients), serving a total of 473,136 clients in 2003. Services are provided based on the needs of the clients -- whether face-to-face or through electronic counseling. Grant funds are exclusive to the development, implementation and delivery of the fundamentals of business management that is relevant to this market.

Evidence: SCORE Daily Desktop Guide; SCORE Operating Manual; SCORE Training Manual; SCORE Marketing Guide; SCORE Fundraising Guide; Cooperative Agreement

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: SBA has developed long-term measures as reflected in the agency's Strategic Plan. In SCORE's five-year plan, the program purpose was refined and performance measures and annual goals supporting the purpose of the program were articulated and disseminated. Performance-based allocation of funding to local chapters is based on chapter achievement against goals. Additionally, SBA has incorporated these long-term measures into its draft research plan which will enable the agency to establish baselines and collect yearly data on goal attainment. The forthcoming Cooperative Agreement for SCORE reflects both long-term and annual goals.

Evidence: The U.S. Small Business Administration Strategic Plan, Fiscal Year 2003-Fiscal Year 2008; SCORE Strategic Plan, FY 2005 SBA Budget Submission and FY 2004 SCORE Cooperative Agreement and the research plan.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Outcome measurements have been developed and are contained in program strategic plans. Goals are structured in a way to achieve program efficiencies concurrent with long-term improvement and program delivery. The program is being challenged to improve program capacity; targets have been set which are ambitious yet achievable utilizing current resources for long-term program delivery. Baselines and refined levels for targets will be further developed with the implementation of the research project in FY 2004.

Evidence: The U.S. Small Business Administration Strategic Plan, Fiscal Year 2003-Fiscal Year 2008; SCORE Strategic Plan and the FY 2005 SBA Budget Submission.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Samples of annual performance criteria in the Notice of Award: SCORE will increase by 3% the number of clients counseled and trained from a baseline of 510,000 clients.

Evidence: Cooperative Agreement and Notice of Award; SBA Scorecard and SCORE Strategic Plan Scorecard.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Baselines have been established for this program beginning with FY 2002 figures. A targeted 3% growth is goaled. Examples include increasing clients served from an actual of 440293 in FY 02 to 462,257 in FY 03. However, new goals based on measurable economic outcomes will be instituted as the research plan develops an effective economic impact measure and test model.

Evidence: Cooperative Agreement; SBA Scorecard and FY 2005 Budget Submission; SCORE Scorecard and Annual Performance Plan; Monthly Chapter Goaling Reports; Quarterly Narrative and Financial Reports; Annual Audit

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: SCORE commits to long-term and annual goals that are negotiated between the National SCORE Association and SBA, provision of which is contained in the annual Notice of Award, the SBA's 5-Year Strategic Plan and the SBA's annual budget request. Annual goals are derived from the FY 2002 baseline and include quantifiable and ambitious targets of increasing both clients served and increasing the diversity in the client base and in the counselor corp. The National Program Manager and the Grants Manager monitor performance through quarterly narrative reports and timelines which include program deliverables (outputs and outcomes).

Evidence: Cooperative Agreement; SBA Scorecard and FY 2005 Budget Submission; SCORE Scorecard and Annual Performance Plan; Monthly Chapter Goaling Reports; Quarterly Narrative and Financial Reports; Annual Audit

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: ED has contracted for services to produce a methodology for program evaluations for all ED programs, including SCORE. The study will commence in June 2004 and a final report will be issued in September 2004.

Evidence: 2004 Contracted Survey; Periodic QSRs (every 2-3 years); CSI Customer Satisfaction Survey.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The SBA FY 2004 budget request directly identifies quantifiable performance targets which were derived from a FY 2002 baseline and the resources necessary for their accomplishment. Future requests for funding are based on performance.

Evidence: Cooperative Agreement; SBA Budget Request and Performance Plan; SCORE Audited Financial Statements; SCORE Annual Report

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The agency has developed a new strategic plan. Likewise, the agency is developing an evaluation strategy that will look at the economic impacts of this and other SBA programs.

Evidence: SBA SCORECARD and Strategic Plan

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: According to the agreement that the SCORE association and SBA make each year, monthly and yearly data is collected. The information is put into management reports.

Evidence: Management Information System (MIS) reports; Quarterly Narrative Reports

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Both the SBA manager and the National Score Association CEO are rated on their performance with respect to accountability for costs and program performance. There is an identified SBA SCORE manager with specific performance elements as well as agency scorecard assignments. For the CEO of NSA, an annual performance plan is in place which includes a baseline salary which can be augmented by performance-based incentives.

Evidence: SBA Performance Review (Pmas); NSA CEO Contract and Performance Plan

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The annual SCORE appropriation is obligated by SBA's Office of Procurement and Grants Management after review and commitment by the program office.

Evidence: Form 1223, Approval documentation, Notice of Award

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Within the SBA strategic plan, SCORE has agreed to increase program efficiency and reduce unit costs by 10% over 5 years. The cost savings of an all volunteer counseling staff is valued conservatively at $65 million -- and all are experienced, qualified business executives. In addition, SCORE uses its internal volunteer expertise to develop and implement projects as well as leverage co-sponsorships and MOUs to augment funding. For example, a former VP of Bechtel led SCORE's development of its electronic database; and the newly launched SCORE website was the result of a partnership with Verizon.

Evidence: SBA Strategic; FY 2005 Budget Request and Performance Plan; SCORE Strategic Plan; SCORE Strategic Partnerships.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Currently, SCORE, SBDCs and Women's Business Centers, and in the BIC work together to assist entrepreneurs and small businesses.

Evidence:  

Does the program use strong financial management practices?

Explanation: Strong financial management controls and procedures are in place. Quarterly financial reports are readily available. Internal and external CPA audits are conducted annually.

Evidence: Audited financial statements. Board of Directors Report. OMB Circular A-110.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program does not have any severe management deficiencies. SBA's CFO reviews the program annually to ensure that the proper cost controls are in place.

Evidence: SBA budget request and Performance Plan, FY2005

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Weekly face-to-face meetings between the program and the agency, weekly memo updates, quarterly reports, and the presence of the SBA Manager at board meetings and annual national meetings informs the agency about grantee activities.

Evidence: MIS Reports; Quarterly Narrative Reports; OBCI Weekly Reports to the Administrator

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The SCORE program collects data monthly, quarterly and annually. The SCORE program is different because of the relationship with the Association. The Association reports annually to a board of directors. This report is available to the public.

Evidence: SBA OIG Inspection Report: "Coordination and Performance Measurement in SBA's Entrepreneurial Development Programs" September 2000, annual report to the Board of Directors

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: ED is conducting a national impact study of all its programs using an approved OMB methodology and survey instruments for SCORE, SBDCs, WBCs and BICs.Results expected in late summer 2004. OMB No. 3245-0351 Expiration 7/31/04

Evidence: 2004 Impact Study. U.S. Small Business Administration Strategic Plan FY 2003 - 2008; SCORE Strategic Plan, FY 2005 Customer Satisfaction Survey, SBA Scorecard and monthly SCORE goal reports.

Does the program (including program partners) achieve its annual performance goals?

Explanation: This program meets all of the annual program performance targets.

Evidence: FY 2003 SCORE Year End Report and SBA FY 2005 Budget Submission.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: SCORE is moving chapters out of government-provided space into low-or no-cost community locations. Expansion of online counseling, now comprisinig 33% of all counseling, was achieved with no additional budget and has contributed substantially to numbers of clients counseled. On-line counseling is available 24 hours a day, 7 days a week, dramatically increasing the convenience and availability of SCORE counseling. Increased recruitment of volunteers has also increased clients counseled with no additional cost. As a volunteer program, it is very cost efficient in achieving annual performance goals with a value of $65 million in volunteer contribution. Furthermore, SCORE is aggressively pushing for new, low-cost or cost-free ways to reach the entrepreneurial market. All leveraged by a $5 million grant. Nonetheless, the agency's internal administrative cost for the program increased significantly in 2001 and 2002 over prior years.

Evidence: FY 2003 SCORE Year End Report, SBA FY 2005 Budget Submission and SCORE Strategic Plan.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: With the implementation of new economic impact studies, new long-term goals will be developed to focus on those outcomes. However, SCORE is SBA's most dynamic and efficient program under current measures. In terms of cost per client, SCORE, at app. $30/client, is much more efficient than any other SBA technical assistance program. Their client counseling numbers have shown significant growth in recent years, and the program has taken a dynamic approach to marketing to entrepreneurs, providing a good counseling product, and cutting costs.

Evidence: GRPA Goaling and Report; SBA Budget Submission and Strategic Plan.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: ED is conducting a national impact study of all its programs using an approved OMB methodology and survey instruments for SCORE, SBDCs, WBCs and BICs.Results expected in late summer 2004. OMB No. 3245-0351 Expiration 7/31/04

Evidence: 2004 Impact Study; SBA Inspector General Report #3-23, April 11, 2003;CommunityPoll Customer Satisfaction Survey, Research Plan and Statement of Work.