Detailed Information on the
Women's Business Centers Assessment

Development of online web tutorial toolkit using URL which can be accessed by grantees for more transparency in grants management processes and procedures. In addition, implementation of HHS payment processing system will allow grantees to access funds within 72 hours. The conversion to HHS will be automated; however during the conversion process, it is currently being done manually.

Business Process Reengineering of the Grant Program for Women's Business Centers: The SBA is reorganizing and redefining of roles and responsibilities of staff that serve the Women's Grant Program, including the Office of Women's Business Ownership (OWBO), the Department of Grants and Procurement Management (DPGM) and District Office Technical Representatives (DOTR). The results of the staffing reorganization are expected to streamline processing of invoice payments, reduce processing time, better apply staff resources, and create more customer-friendly servicing of grants.

Measure: Number of clients counseled

Explanation:Program goals are first negotiated at the local center level between the WBC director and the district office, based on market conditions, budget and other relevant considerations. These goals are then presented, by way of annual proposal, to the SBA's Office of Women's Business Ownership for final review and approval. These numbers are then aggregated to develop a national program goal. 2005 was a new baseline year due to new division-wide client definitions.

Measure: Number of attendees trained

Explanation:Program goals are first negotiated at the local center level between the WBC director and the district office, based on market conditions, budget and other relevant considerations. These goals are then presented, by way of annual proposal, to the SBA's Office of Women's Business Ownership for final review and approval. These numbers are then aggregated to develop a national program goal. 2005 was a new baseline year due to new division-wide client definitions.

Measure: Number of counseling hours

Explanation:Program goals are first negotiated at the local center level between the WBC director and the district office, based on market conditions, budget and other relevant considerations. These goals are then presented, by way of annual proposal, to the SBA's Office of Women's Business Ownership for final review and approval. These numbers are then aggregated to develop a national program goal.

Measure: Number of training hours

Explanation:Program goals are first negotiated at the local center level between the WBC director and the district office, based on market conditions, budget and other relevant considerations. These goals are then presented, by way of annual proposal, to the SBA's Office of Women's Business Ownership for final review and approval. These numbers are then aggregated to develop a national program goal.

Measure: Number of jobs created

Explanation:Information is collected by the WBCs from their clients and reported to the SBA. In FY 2005, we are beginning to collect information on this measure through EDMIS for counseling clients. Program goals are first negotiated at the local center level between the WBC director and the district office, based on market conditions, budget and other relevant considerations. These goals are then presented, by way of annual proposal, to the SBA's Office of Women's Business Ownership for final review and approval. These numbers are then aggregated to develop a national program goal. 2005 was a new baseline year due to new division-wide client definitions.

Measure: Number of small businesses started

Explanation:Information is collected by the WBCs from their clients and reported to the SBA. In FY 2005, we are beginning to collect information on this measure through EDMIS for counseling clients. Program goals are first negotiated at the local center level between the WBC director and the district office, based on market conditions, budget and other relevant considerations. These goals are then presented, by way of annual proposal, to the SBA's Office of Women's Business Ownership for final review and approval. These numbers are then aggregated to develop a national program goal. 2005 was a new baseline year due to new division-wide client definitions.

Measure: Unit cost

Explanation:Cost of provided technical assistance services per client.

Is the program purpose clear?

Explanation: The program is designed to provide technical assistance to women that own or plan to start businesses.

Evidence: Evidence: The Women's Business Ownership Act of 1988 (15 U.S.C. 711(a)) and the Small Business Act (15 U.S. C. 656(k))

Does the program address a specific and existing problem, interest, or need?

Explanation: Women face barriers to starting and growing successful businesses and to accessing capital and procurement opportunities. In 1972, women owned less than 5 percent of all businesses in the United States. Today, as the fastest-growing segment of the economy, they own roughly a third and are represented in all industrial categories. However, women-owned businesses are overwhelmingly sole proprietorships (in 2000, 86 percent of women-owned businesses were sole proprietorships, compared to 73 percent of businesses owned by men), most often represent second jobs, and continue to have significantly lower incomes. According to the U.S. Census Bureau, women-owned firms accounted for just 6.5 percent of employment and only 4.2 of all business receipts. The greatest opportunity gap is among women who are socially and economically disadvantaged, and these make up about half of the WBC clientele. The ability to target this population is among the rating criteria for grants and is part of the SBA's strategic plan for WBCs. It is the niche the WBC Program serves.

Evidence: Survey of Business Owners - Women-Owned Firms: 2002, U.S. Census Bureau; Policy and Progress, Supporting the Growth of Women's Business Enterprise, National Women's Business Council, May 2004; Analyzing the Economic Impact of the Women's Business Center Program, NWBC 2004; U.S. Sole Proprietorships: A Gender Comparison, 1985-2000, Dr. Ying Lowrey, Small Business Administration Office of Advocacy; Enabling the Establishment and Vitality of Small Businesses (SBA Strategic Plan 2003-2008); Impact Study of Entrepreneurial Development Resources, Concentrance Consulting, 2005.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Thw Women's Business Center program has the same general purpose as the Small Business Development Center program (SBDCs); it provides counseling and training to entrepreneurs. However, studies show that women's business centers do not excessively compete for clients with SBDCs or other SBA resource partners, but serve a niche. The WBC Program is the only Federally funded program designed specifically to serve women entrepreneurs and that targets women who are socially and economically disadvantaged. WBCs focus primarily on successfully starting and growing women-owned businesses. And while there are shared practices that underlie the WBCs' success, each is unique and defines its own audience in its grant proposal; most target specific communities or populations. As a result, the program serves a far higher ratio of women of color (more than half) than are represented in the general population (about a third), and clients tend to have lower incomes than the clients of other resource partners. WBC services may range from the most basic, entry-level training (e.g., financial and time-management skills) to assistance in accessing venture capital and managing rapid growth.

Evidence: Survey of Business Owners - Women-Owned Firms: 2002, U.S. Census Bureau The Performance, Progress and Promise of Women's Business Centers in the United States, Womenable, 2006; The Impact and Influence of Women's Business Centers in the United States, Babson College Center for Women's Leadership, 2005; Launching Women-Owned Businesses: A Longitudinal Study of Women's Business Center Clients, Center for Women's Business Research, 2004; Analyzing the Economic Impact of the Women's Business Center Program, National Women's Business Council, 2004; Impact Study of Entrepreneurial Development Resources, Concentrance Consulting, 2005; grant proposals, WBC Program annual reports.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Because of its legislated structure and funding, the WBC Program is complex. There is a constant flux of new centers entering the program and older centers graduating to sustainability or aging out of the program. In addition, the program continues to be adjusted, almost annually, by Congress, and sustainability has been extended on an annual basis for several years.

Evidence: Women's Business Center Program Legislative and Funding History; SBA Congressional Submission Fiscal Year 2007, Performance Budget; P.L. 100-533, Women's Business Ownership Act of 1988; P.L. 102-191, Women's Business Development Act of 1991; P.L. 103-403, Small Business Reauthorization Act of 1994; P.L. 105-135, Small Business Reauthorization Act of 1997; P.L. 106-17, Women's Business Center Amendments Act of 1999; P.L. 106-165, Women's Business Centers Sustainability Act of 1999.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Proposals for both "regular" and sustainability grants undergo a competitive screening process which ranks applicant organizations on their experience in providing or upgrading women's business skills, their ability to commence a project quickly, their location (e.g., areas of greatest need and whether the target audience is already being served by another organization), and their ability to provide training and services to "a representative number of women who are both socially and economically disadvantaged." In addition, the WBC Program uses a performance-based funding formula to allocate resources.

Evidence: WBC Program Announcement and Notice of Award; P.L. 100-533, Women's Business Ownership Act of 1988 (as amended); Grant Proposal Technical Evaluation Plan; Performance-Based Funding Criteria.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The WBC Program is mandated to report on economic impact data, including the program's two key outcome indicators: 1) the number of start-up businesses formed and 2) the number of jobs created. The program's mandate is to work with socially and economically disadvantaged women to assist them in starting, owning and running their own business. The most meaningful long-term outcome measures are therefore business starts and jobs created. Women's business centers are funded for two 5-year program intervals: 1) new (regular) WBCs and 2) sustainability WBCs. The first WBC proposal to SBA, at the beginning of each interval, must contain a 5-year projection of both key output and outcome indicators. As each WBC progresses through each 5-year interval, the program office can not only monitor the center's annual performance, but also compare its actual performance against its 5-year projection.

Evidence: Annual WBC Program Announcement (federal) and Notice of Award; individual WBC proposals (annual with 5-year projections); quarterly WBC reports, annual reports; annual WBC client survey.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has two specific long-term outcome goals??jobs created and businesses started??for which new baselines were established in 2005 with the implementation of the new client definition parameters for all Office of Entrepreneurial Development (OED) programs (although the WBC Program has been collecting this data since the programs inception). As illustrated in 2.1, at the beginning of each funding interval, WBCs summit 5-year projections for outputs and outcomes. These projections are first negotiated at the local level between the WBC and the district office, and are based on local market conditions, funding and other relevant factors.

Evidence: WBC annual Program Announcements, 2004, 2005, 2006, 2007; SBA Congressional Submission Fiscal Year 2007, Performance Budget; Annual WBC Notice of Award; quarterly WBC reports, annual reports; annual WBC client survey.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The WBC Program has a limited number of annual output targets for providing counseling and training to nascent/start-up entrepreneurs and to existing small businesses. These output targets are negotiated at the local level and proposed to the SBA program office. This bottom-up strategy, in addition to the historical "unofficial 3 percent-per-year increase for its performance measures" that the program office has used, has yielded a well-based and accurate approach to predicting performance. The WBCs have consistently exceeded that goal. In addition, the OED program impact study shows that the WBC Program has the highest rating of all SBA technical-assistance programs for customer satisfaction, a significantly higher start-up ratio, higher ratios of job retention and formation, and a significantly higher ratio of increased sales.

Evidence: FY 2005 Performance and Accountability Report; Impact Study of Entrepreneurial Development Resources, Concentrance Consulting, 2005; annual WBC proposals.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The WBC Program established new baselines for all of its annual output measures in 2005, due to the change in client definitions for all Office of Entrepreneurial Development (OED) programs. To that end, the centers continue to coordinate with the district directors to establish ambitious targets for the district offices. Using the Entrepreneurial Development Management Information System (EDMIS), the new division-wide data collection system implemented in October of 2006 and as part of the ongoing Impact Study of Entrepreneurial Resources, the WBC Program now has consistent methods with which to gather output measure data, providing a more valid database of information on which to analyze, compare and project performance.

Evidence: Quarterly data reports; Impact Study of Entrepreneurial Development Resources, Concentrance Consulting, 2005; SBA Congressional Submission Fiscal Year 2007, Performance Budget; FY 2005 Performance and Accountability Report.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The WBCs are funded under cooperative agreements that require them to support the SBA's overall annual performance goals and report their activities related to those goals quarterly through EDMIS. The WBC Program legislation also requires the program to collect economic-impact data from its centers and to submit an annual report to Congress.

Evidence: Women's Business Ownership Act of 1988; WBC Program Announcement and Notice of Award; Annual Report to Congress; quarterly data reports.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: A number of independent studies have been conducted, some of which are longitudinal studies (three of those are summarized in "The Performance, Progress and Promise of Women's Business Centers in the United States," a research analysis report). In addition, the SBA's Office of the Inspector General conducted audits of certain centers that the program identified as problematic. In response to the findings, the program office instituted improved oversight procedures. A successful audit by the General Accounting Office also led to enhanced procedures.

Evidence: The Performance, Progress and Promise of Women's Business Centers in the United States, Womenable, 2006; The Impact and Influence of Women's Business Centers in the United States, Babson College Center for Women's Leadership, 2005; Launching Women-Owned Businesses: A Longitudinal Study of Women's Business Center Clients, Center for Women's Business Research, 2004; Analyzing the Economic Impact of the Women's Business Center Program, National Women's Business Council, 2004; GAO audit report; OIG audit reports on WBCs.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The SBA budget submissions clearly present measurable performance targets and the resources necessary for their accomplishment. The agency uses an activity based cost model to determine full program costs.

Evidence: SBA Congressional Submission Fiscal Year 2007, Performance Budget.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program office re-evaluates its strategic planning in light of frequent changes in legislation, the WBC Program's greatest controlling factor. Most significantly, it is addressing the future needs of the WBCs, many of which will soon leave the program due to completing their final eligible year of either regular or sustainability funding. Office of Womens Business Outreach (OWBO) has added training and mentoring plans to help prepare these WBCs to continue to operate without federal funding.

Evidence: Enabling the Establishment and Vitality of Small Businesses (SBA Strategic Plan 2003-2008); internal strategic planning memos.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: OWBO collects quarterly training and counseling statistics, as well as quarterly performance and financial reports from the WBCs. OWBO also requires semiannual programmatic and financial reviews, which are conducted, often on site, by District Office technical representatives (DOTRs), as well as bi-annual, independent financial reviews. Each WBC's records must also be reviewed annually by a certified public accountant.

Evidence: Quarterly reports; semiannual programmatic and financial reviews; independent financial reviews; WBC Program Announcement and Notice of Award.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: District Office technical representatives (DOTRs) directly oversee the individual WBCs and conduct regular, often on-site, reviews; address issues as needed; and review and channel pay requests and other required documents to OWBO. OWBO reviews WBC documentation for compliance and correctness; forwards corrected, completed pay request packages to OPGM for processing; and works with the district offices to address issues as needed. Grant recipients undergo semiannual and biannual reviews, as detailed in 3.1. In addition, the WBCs do not receive advances or reimbursements unless they meet program requirements and submit required data and correct, complete paperwork.

Evidence: WBC Program Announcement and Notice of Award; grants management manual.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Funds are obligated in a timely manner after review and selection of grantees. Once proposals are approved and funding agreements established for each center, a national, mandatory post-award training conference is held to train field and WBC staff on grant management and program oversight procedures; management of the financial aspects of the grant, including allowable expenses, documentation required, etc.; and program goals, guidelines and performance requirements. Oversight is conducted monthly via conference calls and more detailed reviews are conducted using the WBCs' quarterly reports and documented invoices.

Evidence: Annual Report to Congress; WBC Program Announcement and Notice of Award; individual WBC records.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Under the SBA's activity costing methodology, the Federal cost of delivering services to WBC clients is measured and tracked. WBC Program grants are awarded on a competitive basis, and applicants must demonstrate their ability to meet the required match. In addition, the WBC Program piloted performance-based funding for grant recipients.

Evidence: SBA's activity based costing system; WBC Program Announcement and Notice of Award; Annual Report to Congress; Grant Proposal Technical Evaluation Plan; Performance-Based Funding Criteria.

Does the program collaborate and coordinate effectively with related programs?

Explanation: WBCs often use SCORE (formerly know as Service Corps of Retired Executives) counselors to train and counsel clients and assist with mentoring roundtables. WBCs sometimes share facilities such as meeting rooms or libraries with Small Business Development Centers (SBDCs), and co-host events with both SBDCs and SCORE. WBCs are required by legislation to obtain matching funds and in-kind contributions from state, local and private organizations. They also leverage their resources by drawing on the community for volunteers to assist with training, counseling, mentoring, legal advice and other services.

Evidence: The Performance, Progress and Promise of Women's Business Centers in the United States, Womenable, 2006; The Impact and Influence of Women's Business Centers in the United States, Babson College Center for Women's Leadership, 2005; Launching Women-Owned Businesses: A Longitudinal Study of Women's Business Center Clients, Center for Women's Business Research, 2004; Analyzing the Economic Impact of the Women's Business Center Program, National Women's Business Council, 2004; WBC Program Announcement and Notice of Award; WBC narrative reports.

Does the program use strong financial management practices?

Explanation: WBCs must submit quarterly performance and financial reports and must undergo semiannual programmatic and financial reviews by DOTRs, as well as bi-annual, independent financial reviews. Each WBC's records must also be reviewed and approved annually by a Certified Public Accountant. Requests for reimbursement or advances must be supported by detailed, reconcilable expenditure reports, which must be complete and correct before being processed.

Evidence: WBC Program Notice of Award; grants management manual (in development); individual WBC records.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Frequent changes by Congress in the program structure and funding ratios, and delayed extensions of sustainability and its uncertain future make program management and analysis??especially strategic and long-term planning??problematic (see 1.4). However, OWBO takes a proactive approach in finding ways to more effectively and efficiently manage the program as it grows. OWBO instituted monthly conference calls with WBCs and district office personnel that include best practices, structured training, guest speakers and problem resolution. Conference call schedules and reporting deadlines are provided on a project-year calendar. OWBO initiated the development of a grants management manual for use by WBC program officers and SBA grant managers to ensure consistency and to address ongoing documentation issues that cause delays with pay requests. As a complement to that, OWBO is also developing a training module and manual for use by the DOTRs and WBCs. In addition, OWBO instituted improved oversight procedures following audits of certain centers by the SBA Office of the Inspector General and a successful audit of the WBC Program by the General Accounting Office.

Evidence: GAO Audit; OIG audits; WBC calendar; grants management manual.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has two specific long-term outcome measures: jobs created and businesses started. In FY 2005 the WBCs reported creating 9,442 jobs and gross receipts from assisted business of $364 million, up from 5,108 jobs and gross receipts of $123 million in 2002. During the same period, number of businesses started increased from 3,592 to 4,295. However, the 2005 numbers use a new baseline and are not directly comparable to earlier years.

Evidence: Individual grant proposals and notices of award; quarterly data submissions; Enabling the Establishment and Vitality of Small Businesses (SBA Strategic Plan 2003-2008); Impact Study of Entrepreneurial Development Resources, Concentrance Consulting, 2005; goaling projections and life-cycle analysis of WBCs (currently being undertaken).

Does the program (including program partners) achieve its annual performance goals?

Explanation: The WBC Program has occasionally missed its annual output goals. Overall, the program met or exceeded all of its 2003 goals. However, in 2004 the program missed two out of 4 annual goals. Beginning with 2005, a new baseline for annual goals has been established.

Evidence: Individual grant proposals and notices of award; quarterly data submissions; Enabling the Establishment and Vitality of Small Businesses (SBA Strategic Plan 2003-2008); Impact Study of Entrepreneurial Development Resources, Concentrance Consulting, 2005; goaling projections and life-cycle analysis of WBCs (currently being undertaken).

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The WBC Program consistently seeks to improve its efficiencies and demonstrate effectiveness. For example, the WBC Program piloted performance-based funding for grants, initially to sustainability centers and then to all WBCs beginning in their second year. OWBO is currently conducting a life-cycle study of WBCs, to assist with future planning and goaling. Results of OWBO's ongoing efforts are evident in the agency's budget, which shows that the cost per client served by the WBC Program in FY 2004 was $177, only $34 more than the SBDCs. In 2005, the cost per client dropped to $163, just $14 more than the SBDCs.

Evidence: Impact Study of Entrepreneurial Development Resources, Concentrance Consulting, 2005; FY 2005 Performance & Accountability Report; SBA Congressional Submission Fiscal Year 2007, Performance Budget; Grant Proposal Technical Evaluation Plan; Performance-Based Funding Criteria; WBC Life-Cycle Study (underway).

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The WBC Program has increased its number of clients served from 45,223 in 2002 to nearly 145,000 clients in 2005, while reducing per client costs. The ongoing impact study of Entrepreneurial Development programs shows that the WBC Program leads on almost every indicator. External studies show a 14:1 ratio of return on the federal investment??between 2001 and 2003, that translated to more $500 million in gross revenues and an estimated $51.4 million in profits from a federal investment of $36.5 million. Studies also show that resource partners??SCORE, SBDCs and WBCs work together, often in coordination with the DOTR/WBOR in the district office, to deliver the services that best meet the needs of each client.

Evidence: Impact Study of Entrepreneurial Development Resources, Concentrance Consulting, 2005; Annual Report to Congress; quarterly data reports; The Performance, Progress and Promise of Women's Business Centers in the United States, Womenable, 2006; The Impact and Influence of Women's Business Centers in the United States, Babson College Center for Women's Leadership, 2005; Launching Women-Owned Businesses: A Longitudinal Study of Women's Business Center Clients, Center for Women's Business Research, 2004; Analyzing the Economic Impact of the Women's Business Center Program, National Women's Business Council, 2004.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Several independent studies have shown the WBC Program to be effective in reaching its audience and positively affecting business startup and growth, job creation and revenues among women entrepreneurs. Most recently, OED's Impact Study of Entrepreneurial Development Resources showed that, in the first year of this longitudinal study, WBCs had an 81 percent satisfaction rate (compared to 70 percent for all ED resources), and an excellent rate of both businesses started (6,600 new firms between 2001 and 2003, a 376 percent increase, and nearly 8,000 more by 2005) and jobs created (more than 12, 000 during the same period and more than 17,000 more by 2005).

Evidence: Impact Study of Entrepreneurial Development Resources, Concentrance Consulting 2005; The Performance, Progress and Promise of Women's Business Centers in the United States, Womenable, 2006; Analyzing the Economic Impact of the Women's Business Center Program, National Women's Business Council, 2004; quarterly data reports.