Detailed Information on the
Minority Business Development Agency Assessment

Explore the feasibility of developing additional long term measures, possibly on survival rate of MBDA-assisted small businesses.

Seek additional efficiencies to reduce the cost per client assisted.

Explore the feasibility of contracting with an independent organization to conduct an evaluation on the long-term impacts MBDA's assistance has on minority businesses.

Measure: Dollar value of contract awards to Minority Business Enterprises (MBEs)

Explanation:MBDA accomplishes its performance goal through the implementation of several business development programs. The success of these programs is measured by the dollar value of contract awards obtained by minority business enterprises (MBEs) and facilitated by MBDA's grantees and staff. The certainty that MBEs will realize the proceeds associated with these awards varies from contract to contract. Multiple year contracts with option years are less certain as the options may or may not be exercised. MBDA includes the full potential value of multiple year contract awards obtained in its annual reporting for this performance measure, and discloses the dollar value of option years in a footnote. For indefinite-delivery contracts, only actual dollar values realized or guaranteed are included in the annual reporting of this performance measure.

Measure: Dollar value of financial awards obtained for clients

Explanation:This represents the cumulative, annual dollar value of transactions that have been approved, verified and validated for each financial package (loans, lines of credit, surety bonds, etc.) obtained for clients serviced by MBDA funded projects, agency staff, or the MBDA Portal on-line tools.

Measure: New jobs created for client businesses

Explanation:The growth in the number of new employees will demonstrate MBDA's long-term success and contribution to the Nation's economy. This measure focuses specifically on the number of jobs created within minority-owned business enterprises as a result of contract and financial services provided by MBDA funded projects and agency staff.

Measure: Increase in client gross receipts

Explanation:MBDA measures increases in MBE gross receipts to determine the growth in firm size to further achieve entrepreneurial parity. This measure focuses specifically on the increase to individual minority business enterprise receipts as a result of the services provided by MBDA's funded projects and staff.

Measure: Satisfaction rating in the American Customer Satisfaction Index (ACSI)

Explanation:Working with the Federal Consulting Group at the Department of Treasury and the University of Michigan, MBDA has developed a program survey module to measure customer satisfaction and has established an American Customer Satisfaction Index. This survey is conducted in odd numbered years.

Measure: Cost per client assisted by MBDA

Explanation:Annual cost is based on number of all clients, including those serviced by grantees and the MBDA portal. This total number of clients is measured against the total cost of MBDA's grants and MBDA's information technology support contract.

Measure: Cumulative Economic Impact

Explanation:In reviewing its annual activities related to the dollar value of contracts and financial awards, the long term goal of achieving $30 billion dollars in cumulative economic impact by 2020 has been established (base year of FY 1999). Progress toward this goal can be gauged on an annual basis.

Is the program purpose clear?

Explanation: The Minority Business Development Agency (MBDA) is a bureau within the United States Department of Commerce. The Agency's mission is to promote the growth and competitiveness of minority business enterprises (MBEs). MBDA's primary objective of promoting growth and competitiveness for MBE's ties to the concept of entrepreneurial parity for MBEs in the economy. Entrepreneurial parity is defined as reaching proportionality between the minority population percentage and the percentage share of business development measures such as numbers of firms, gross receipts and employment.

Evidence: MBDA mission statement; Executive Order 11625 (http://www.mbda.gov/?section_id=2&bucket_id=643&content_id=3147&well=entire_page).

Does the program address a specific and existing problem, interest, or need?

Explanation: MBDA was established to address the demands and barriers experienced by minority-owned firms and entrepreneurs for the purpose of gaining full access and participation in the National economy. According to the 2002 Survey of Business Owners (SBO), although minority-owned firms are increasing in number at a faster pace than the rate of the minority population growth, the gap in annual gross receipts slightly widened when compared to the share of the minority population. If, however, the number of minority firms reflected the 2002 minority population share, there would have been over 2.4 million more firms, totaling over 6.5 million firms. Gross receipts would have been more than $2.5 trillion, which is almost four times the actual number. Paid employment would have totaled nearly 16.1 million??more than three times the actual number.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm); Executive Order 11625 (http://www.mbda.gov/?section_id=2&bucket_id=643&content_id=3147&well=entire_page); State of Minority Business Enterprises: An Overview of the 2002 Survey of Business Owners (http://www.mbda.gov/minoritybizfacts/StateofMinorityBusiness2002SurveyofBusinessOwners.pdf); The State of Minority Business (All Minority Groups) Fact Sheet (http://www.mbda.gov/minoritybizfacts/?bucket_id=789)

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: While MBDA is focused on minority small businesses, there many other sources of business assistance and support at the Federal, state, and local level.

Evidence: Federal sources include a number of business assistance programs in the U.S. Small Business Administration (SBA), in addition to state and local business development programs.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: MBDA operates a network of nearly 50 centers, or grantees, around the country and Puerto Rico. These centers provide business consulting services to the minority business community, especially those entrepreneurs who are pursuing accelerated growth. MBDA's service delivery model of having grantees operate centers on the agency's behalf is useful because local entities in the minority business community are the most knowledgeable about the needs of MBEs in their designated areas. MBDA's competitive grant programs require that applicants qualify for awards based on their proposed performance results and overall merit of application. Client services are regularly monitored by MBDA to ensure adherence to agreed upon standards and to ensure performance results. In pursuit of entrepreneurial parity, MBDA has engaged in a Strategic Growth Initiative (SGI), which is designed to address the issue of sustainable business value for firms of size operating in high-growth industries.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm) Published Federal Funding Opportunities for MBDA grant programs (http://www.grants.gov)

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Minority Business Enterprises (MBEs) are defined as minority-owned businesses. MBDA ensures that intended beneficiaries receive services by identifying specific markets for locating grant funded centers. These determinations are based on the size and ranking of the minority population in those markets, the overall number of minority firms, as established by MBDA and U.S. Census Bureau data, as well as other available information about the Nation's minority population. MBDA programs also require grantees to provide matching funds and to develop and build upon strategic alliances with public and private sector partners. These partnerships help the centers to serve SGI firms and the growing numbers of MBEs with rapid growth-potential within the grantee's geographic service area.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm) Published Federal Funding Opportunities for MBDA grant programs (http://www.grants.gov)

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: MBDA's performance goal is to increase access to the marketplace and financing for minority-owned businesses. MBDA has developed long-term performance measure for cumulative economic impact, and progress towards that goal can be tracked annually. MBDA has set the long-term goals of obtaining $30 billion dollars in cumulative contracts and financials, cumulative economic impact by 2020.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: MBDA's long-term targets are developed based on an extrapolation of its annual targets for contracts and financials. Given the difficulty of separating out the impact of MBDA assistance from overall economic trends, it is useful to focus on MBDA's specific cumulative economic impact. However, a straight-line projection of annual targets, summed every five years, is not considered ambitious, and does not necessarily indicate thorough strategic planning. Developing a measure that examines the survival rate of MBDA-assisted firms, years after assistance is provided, would enhance MBDA's strategic planning process. Comparing the survival rate of MBDA-assisted businesses to the national average for comparable businesses would provide a strong link towards long-term, program performance.

Evidence: MBDA Annual Performance Plan, FY 2006 Commerce Performance and Accountability Report (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: MBDA has five annual performance measures: 1) new jobs created for client businesses, 2) percent increase in client gross receipts, 3) dollar value of contract awards to MBEs, 4) dollar value of financial awards obtained for clients, and 5) percent increase in the American Customer Satisfaction Index. These annual measures support MBDA's long-term measures related to economic impact and new job opportunities.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Generally, the targets set for MBDA's measures are ambitious. For most previous fiscal years, MBDA has been able to meet the targets set. Targeting for the measures dealing with the dollar value of contracts and financials is now well established and MBDA has a good understanding of its capabilities to achieve its targets in these areas. The jobs created measure was implemented in FY 2005, based on a number of variables including the cyclical nature of the grantees, the overall number of grantees that contribute to agency performance, and uncertainties with the economy as a whole. Based on performance in FY 2006, MBDA has increased its annual target to 3,000 jobs created. MBDA should reevaluate this target as more peformance data is realized.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: MBDA grantees' specific performance goals match those of MBDA and annual performance measures and targets for each grantee are a primary aspect of the grant application, award, and monitoring processes. MBDA's service delivery model is based on grantees providing the majority of agency services to MBEs. Therefore, MBDA grantees are committed to and work toward MBDA's annual and long-term performance goals.

Evidence: MBDA receives performance data from grantees weekly and is reported quarterly and annually. This data is found in MBDA grant monitoring and program management reports, and Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: While MBDA has developed performance measures, including one measuring client customer satisfaction, there have not been any recent, regular independent program evaluations that meet the criteria for a Yes for Question 2.6. MBDA should contract with an independent organization to conduct an evaluation on the long-term impacts MBDA's assistance has on minority businesses.

Evidence: SBA's Small Business Development Center (SBDC) program has had a number of independent evaluations conducted looking at the long-term impact of their assistance on firms, evaluations that could also be beneficial for MBDA.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: MBDA programs as a whole are aligned with MBDA's performance goals. The MBDA annual performance plan is integrated with its budget request, and all direct and indirect costs are tied to agency needs. The annual MBDA budget justification directly ties the funding requested to MBDA's performance goals and measures. Annual reporting of full grantee costs are made through established reporting mechanisms and reviewed by the grants officer in the Department of Commerce's Office of the Secretary.

Evidence: FY 2008 MBDA Congressional Budget Request, MBDA Annual Performance Plan

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: MBDA has refined its outcome performance measures and integrated its day to day management with its strategic goals, objectives, and performance measures. The MBDA strategic plan is developed by senior management at quarterly retreats. It includes Agency priorities and goals to deliver performance. The staff performance plans are also synchronized with each agency goal and measure. Also included are Long-term goals, management challenges, and best practices.

Evidence: MBDA Strategic Plan (http://www.mbda.gov/index.php?section_id=6&bucket_id=16&portal_document_download=true&download_cid=6057&name=MBDA+Strategic+Plan+for+FY+2007-+FY+2010.doc&legacy_flag=false) MBDA Annual Performance Report (http://www.mbda.gov/?section_id=6&bucket_id=120&content_id=5018&well=entire_page) Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: MBDA grantees report performance data on a daily basis and MBDA regional staff monitor progress and ensure performance data validity through established verification techniques. MBDA program managers, with the MBDA management team, review performance data for the entire agency on a weekly basis. Progress is monitored and when poor performance is detected, intervention strategies are developed and implemented. All grantees maintain a pipeline of contracts and financials that is termed "work-in-progress." MBDA staff monitors this data and assists when needed to see that the performance is achieved. MBDA reports performance data quarterly to senior Department of Commerce officials and review progress toward annual performance goals. Regular reviews of performance data ensure that grantees meet or exceed performance targets and help to improve overall program design by informing design changes reflected in new grant cycle competitions.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm); Published Federal Funding Opportunities for MBDA grant programs (http://www.grants.gov); MBDA Automated Performance System Reports

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Performance data for regional offices is used in evaluating the performance of the regional directors. Regularly collected performance data measures each director's success in monitoring grantee progress toward meeting or exceeding annual performance goals and thus agency performance goals. MBDA ensures rigorous oversight of staff and grantee due diligence through regular communication, data validation and verification, and when needed, the development and implementation of intervention strategies to improve poor performance. MBDA program managers work with legal counsel and grant officers to discontinue funding a poor performing grantee when all other avenues to assist and improve performance have been pursued without adequate results.

Evidence: Performance evaluations for the National Director and the Regional Directors reflect their performance against established targets. MBDA Annual Performance Report (http://www.mbda.gov/?section_id=6&bucket_id=120&content_id=5018&well=entire_page); MBDA grant termination documents from FY 2006.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: MBDA typically awards and obligates its grants in the second quarter of each fiscal year. All MBDA funds are obligated annually within established spending plans. Grants are obligated at time of award and grantees draw down on these funds under the supervision of the grants officer in the Department of Commerce's Office of the Secretary. Actual expenditures are reported via established reporting mechanisms and reviewed by the grants officer. As awards are made, obligations are immediately reported in the financial system of record. MBDA has had no Anti-Deficiency Act violations and typically closes each fiscal year with a very limited amount of unobligated funds.

Evidence: Department of Commerce quarterly financial statements; Department of Commerce and MBDA budget and performance documents, (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: MBDA has achieved efficiencies through outsourcing information technology services and by outsourcing other support services such as accounting, human resources, and procurement to other Commerce bureaus. MBDA has also developed a cost per client assisted measure.

Evidence: FY 2006 Department of Commerce Performance and Accountability Report; MBDA budget and grantee performance data.

Does the program collaborate and coordinate effectively with related programs?

Explanation: MBDA regularly coordinates with other Federal agencies, including SBA, the Department of Agriculture, the General Services Administration, and various state and local agencies. For example, MBDA has worked diligently with other Federal agencies and state and local governments to ensure minority participation in the post-2005 Gulf Coast hurricane recovery effort. MBDA has also had a long partnership with the Amos Tuck School of Business at Dartmouth College. The Tuck program has trained over 400 business consultants and staff to-date and has provided a curriculum to meet the challenges of a rapidly growing minority population with strategies to better serve MBEs. Major alliances have also been established with private sector organizations and firms such as the Kauffman Foundation, Microsoft, Lockheed Martin; and with National Chambers of Commerce to include the U.S. Hispanic Chamber, the National Black Chamber, the National Urban League, and the U.S. Pan Asian Chamber. All MBDA partnerships support the agency mission of promoting the growth and competitiveness of MBEs.

Evidence: MBDA Business to Business Linkages Forums held in New Orleans and Biloxi. MBDA Memoranda of Understanding (MOUs) with other Federal agencies, state and local governments, private sector firms, and non-government organizations (NGOs). Tuck School - Executive Education Program (http://www.tuck.dartmouth.edu/exec/)

Does the program use strong financial management practices?

Explanation: Evidence of effective financial management is represented by the Department of Commerce's unqualified financial opinions on its financial statements. The National Institute of Standards and Technology (NIST) provides accounting services for MBDA. MBDA management is regularly updated on the status of obligations, consistent with the financial system of record, which allows for informed decision-making. MBDA is also a full participant in the Department's A-123 efforts to ensure that strong internal controls are in place and fully implemented. MBDA has fully documented its processes and has tested those processes without significant findings.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Has the program taken meaningful steps to address its management deficiencies?

Explanation: MBDA has streamlined its organizational structure to achieve operational efficiencies. MBDA is also in the process of improving operational and administrative processes through the implementation of a customer relations management improvement effort. Further, in response to a 2005 Office of the Inspector General Report, MBDA has also addressed management deficiencies with its grant programs and performance data collection efforts. All MBDA grant programs are now monitored on a consistent basis with a refined and updated monitoring handbook. Definitions for grantees related to reporting performance have also been updated to ensure consistent reporting, in line with accepted performance measurement and reporting practices. MBDA data systems have also been refined to reflect these changes and to ensure consistent performance reporting.

Evidence: MBDA Annual Financial Management Guides, Customer Relations Management improvement reports, American Customer Satisfaction Index reports. DOC OIG Report, Value of MBDA Performance Measures is Undermined by Inappropriate Combining of Program Results and Unreliable Performance Data From MBOC Program - Performance Audit, (http://www.oig.doc.gov/oig/reports/2005/MBDA-FSD-17252-5-0001-09-2005.pdf)

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: MBDA conducts a competitive grant solicitation process whereby the evaluation criteria are fully disclosed via grants.gov and the Federal Register. The evaluation criteria explicitly identify how achieving performance results is a significant requirement for obtaining and continuing Federal funding. Each applicant is fully aware that Federal funding is contingent upon meeting or exceeding performance goals as published. Successful grant applicants sign three-year cooperative agreements that are renewed annually. Grant renewals are made only upon completion of a thorough and successful evaluation of a grantee's performance.

Evidence: Published Federal Funding Opportunities for MBDA grant programs (http://www.grants.gov)

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The agency has 5 regional offices and over 50% of the staff is located in the regions. MBDA staff review and evaluate grant applications and also conduct day-to-day grant monitoring functions. MBDA has implemented rigorous performance data validation and verification procedures for regional staff to perform while monitoring grantees. Grantees submit comprehensive quarterly reports and submit performance data into an electronic tracking system on a daily basis. MBDA program managers, with the MBDA management team, review performance data for the entire agency on a weekly basis. Progress is monitored and when poor performance is detected, intervention strategies are developed and implemented. Regional management and business development specialists are held accountable for the grants that they are responsible for monitoring. MBDA managers also keep track of "work-in-progress" to ensure performance goals are met. All grantees maintain a pipeline of contracts and financials that is termed "work-in-progress." MBDA staff monitors this data and assists when needed to see that the performance is achieved.

Evidence: MBDA grant application files, MBDA grant monitoring handbook, and individual staff performance plans.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: MBDA grantee performance data is aggregated to report overall agency performance annually. This data is available in the Department of Commerce Annual Performance and Accountability Report and in MBDA annual budget justifications. Both of these documents are available on the internet on the DOC website. The MBDA annual performance report also provides annual performance data by grantee and is publicly available on the internet.

Evidence: Department of Commerce and MBDA budget and performance documents, (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm); MBDA Annual Performance Report, (http://www.mbda.gov/?section_id=6&bucket_id=120&content_id=5018&well=entire_page)

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: MBDA has identified its cumulative economic impact as its primary long-term performance measure. Each year since FY 1999, with the exception of FY 2003, MBDA has exceeded its combined goal for contracts and financials. However, while focusing on cumulative economic impact is useful in that it strongly links MBDA assistance to tangible results, a summation is not considered ambitious, and does not provide any additional insight into how MBDA's activities are impacting economic parity nationally or in a specific region, or improving the survival rate of assisted minority businesses. MBDA should explore the feasibility of developing measures that track the long-term impacts of MBDA's activities on the firms they assist nationally and possibly within the regions where services are provided.

Evidence: MBDA Annual Performance Plan, FY 2006 Department of Commerce Performance and Accountability Report, (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Does the program (including program partners) achieve its annual performance goals?

Explanation: In FY 2005, MBDA met or exceeded all its performance measure targets. In FY 2006, MBDA met or exceeded all targets except one. MBDA did not achieve its FY 2006 target for dollar value of financial awards due to an unforeseen policy change. In FY 2006, MBDA was directed to ensure minority participation in the post-hurricane gulf recovery effort. In order to support the President and the Secretary of Commerce in this effort, MBDA put significant resources toward MBEs obtaining contracting opportunities. The target of $0.45 billion for financial awards was not achieved in FY 2006, however, MBDA did achieve $0.41 billion for this performance measure.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm)

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: MBDA has streamlined its organization through increased use of information technology systems. MBDA has also developed a cost per client served measure to determine program efficiencies and cost effectiveness. However, MBDA's cost per client is four times higher than the cost per client served of similar programs in the U.S. Small Business Administration. While this comparison may be imperfect, due to different methodologies, it does indicate that MBDA should investigate further efficiencies.

Evidence: MBDA Annual Performance Plan (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm); MBDA budget and grantee performance data. The cost per client for the Small Business Administration's Small Business Development Center (SBDC) and Women's Business Center programs range between $150 and $200.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: MBDA's annual performance measures do compare favorably with similar technical assistance programs such as the Manufacturing Extension Partnership (MEP), the Small Business Development Center (SBDC) program, and the Women's Business Center (WBC) program. However, beyond performance measures, more rigorous research and evaluation needs to be conducted to better understand how MBDA's program compares to and compliments other Federal, state, and local business development and technical assistance programs and efforts.

Evidence: Department of Commerce and MBDA budget and performance documents (http://www.osec.doc.gov/bmi/budget/budgetsub_perf_strategicplans.htm); PART review for MEP (http://www.whitehouse.gov/omb/expectmore/summary/10000040.2002.html); PART review for SBDC (http://www.whitehouse.gov/omb/expectmore/summary/10000368.2004.html)

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: There have not been any independent program evaluations of sufficient scope or quality conducted. MBDA should explore the feasibility of contracting with an independent organization to conduct an evaluation on the long-term impacts MBDA's assistance has on minority businesses.

Evidence: No evaluations of sufficient scope have been conducted to meet the quidance for Questions 2.6 and 4.5.