Detailed Information on the
Historically Underutilized Business Zone - HUBZone Assessment

Review 5% of portfolio (as of 10/01/2007) for compliance with program requirements.

Improve the timeliness of the decertification of ineligible firms.

Undertake evaluations to measure program outcomes

Review 5% of portfolio annually for compliance with program requirements.

Measure: Program certification actions

Explanation:This output measure reflects the number of new firm certified under the HUBZone Program each year.

Measure: Program examinations completed.

Explanation:This output measure reflects the number of examinations completed each year to determine compliance of a 5% sampling of HUBZone-certified firms to determine compliance with program requirements.

Measure: % customer satisfaction.

Explanation:This outcome measures the proportion of firm certified as 'qualified HUBZone small business concern' expressing satisfaction with HUBZone Program customer services.

Measure: Cost per small business certified.

Explanation:

Measure: Capital investment increase.

Explanation:This outcome measure reflects the net increase in capital investments of 'qualified HUBZone small business concerns' attributable to ceretification of firms as such.

Measure: Jobs created.

Explanation:This outcome measure reflect the level of employment of HUBZone-ertified firms.

Is the program purpose clear?

Explanation: The Historically Underutilized Business Zones or HUBZone Program stimulates capital investment, creation of jobs, and economic development in distressed rural and urban communities by providing Federal contracting preferences to firms certified by SBA as 'qualified HUBZone small businesses.'

Evidence: HUBZone Act of 1997, HUBZones in Native America Act of 2000, Small Business Administration Authorization and Manufacturing Assistance Act of 2004, and program regulations (13 CFR 126.100)

Does the program address a specific and existing problem, interest, or need?

Explanation: This program addresses chronic economic development issues in urban and rural communities. HUBZone certification helps small businesses in distressed communities secure federal contracts, which in turn helps create jobs, encourages capital investment, and spurs the local economy. HUBZones include: qualified census tracts in urban areas which qualify for low income housing tax credits; non-metropolitan counties in which average household income does not exceed 80% of the state-wide average, or in which the average level of unemployment is not less than 140% of the lower of the national or state-wide average; Indian Lands; and areas affected by Department of Defense base closures and realignments. These areas are by definition the most economically distressed communities in the Nation.

Evidence: Statistics collected and published by the Departments of Housing and Urban Development, and Bureaus of Labor Statistics and Census. Areas affected by base closures and realignments are published on an ad hoc basis by the Department of Defense. Indian Lands are determined as necessary by the Bureau of Indian Affairs. Presently approximately 11,600 qualified census tracts in metropolitan areas and 1,400 non-metropolitan counties are eligible under the HUBZone Program. Further, approximately 150 communities affected by base closures and realignments qualify for the program.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program is not excessively duplicative or redundant of other procurement preference or economic development program. The HUBZone Program is geographically based and community centric while other preference programs e.g., small disadvantaged business (SDB), and 8(a) Business Development (8(a)) are firm-centric. A key objective of the 8(a) Program is to promote capital formation among firms owned and controlled by socially and economically disadvantaged individuals. Certification hinges on the economic and demographic characteristics of firm principals. The program seeks to provide for business development of such firms through access to sole-source and limited-competition contracting opportunities, and provision of management and technical assistance. Parallel eligibility requirements are imposed upon firms seeking designation as SDB's. The HUBZone Program promotes job growth, investment, and economic development in distressed urban and rural communities. The program seeks to achieve its objective through provision of access to contract opportunities, substantially through full and open competition with application of a price evaluation adjustment, and through competition among HUBZone firms. Certification under the program hinges on the characteristics of the firm's location and its employment patterns. The firm's principal office must be locate in a distressed community, and at least 35% of employees must be residents of such communities. Nonetheless, firms may be certified under 8(a), SDB, and HUBZone Programs. As of May 31, 2005, 12,983 firms were HUBZone-certified. Of this number, 17%, were SDB-certified, including 13%, which were 8(a) certified.

Evidence: Regarding HUBZone, 8(a), and SDB Programs, Small Business Act, as Amended, and relevant program regulations.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program design does not have major flaws that would limit its effectiveness or efficiency. The General Accountability Office assessed the program in FY 2001, and found that acquisition agencies were not accurately characterizing contract awards under the program, which could result in misstatement of results. This circumstance has been partially mitigated by issuance of procedural guidance by the Office of Federal Procurement Policy (OFPP) regarding coding of awards to HUBZone-certified firms. It can be further mitigated by increased reliability and timeliness of data contained in the Federal Procurement Data System-Next Generation (FPDS-NG). While this problem is a cause for concern, it is not a design flaw regarding the program. In January 2003, the Office of the Inspector General issued Audit Report Number 3-15, "HUBZone Companies and a Review of the HUBZone Empowerment Contracting Program's Internal Controls." The report recommended that the Agency develop and fund a plan to conduct an adequate number of program examinations to protect program integrity and lessen opportunities for contracting fraud. The Agency rectified the internal control weakness by developing and implementing an efficient and rigorous program examination process. The process provides for annual review of at least 5% of the program's portfolio. By law, Native American Lands are prima facie geographically eligible under the HUBZone Program. It is possible that in limited circumstances, Native American Lands that were not truly economically distressed would qualify under the program. In such cases, relatively affluent communities, and small firms located therein, might benefit, despite program intent to the contrary. This potential risk could be mitigated through application of a dual geographical eligibility test. That is to be geographically eligible under the HUBZone Program, Native American Lands would have to qualify as areas of high unemployment, low income, or as qualified census tracts.

Evidence: Report GAO-02-57, HUBZone Program Suffers from Reporting and Implementation Difficulties; Office of the Inspector General Audit Report Number 3-15, "HUBZone Companies and a Review of the HUBZone Empowerment Contracting Program's Internal Controls."

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The HUBZone Program is targeted to provide Federal contract opportunities - revenue sources - to firms located in approximately 11,600 metropolitan census tracts, 1,400 non-metropolitan counties, Indian Lands, and 150 areas affected by base closures and realignments. These are areas that suffer high unemployment and/or low household income. To be eligible to participate in the program, among other things, a firm's 'principal office', the place at which the greatest number of non-contract-specific employees routinely perform their work, must be located in a HUBZone and at least 35% of its employees must be residents of HUBZones (on average, approximately 70% of the employees of HUBZone-certified firms reside in such communities). As a result of Inspector General findings (report 3-05) that some certified firms were no longer qualified for the program, the office is implementing a review process that will review 5% of participating companies annually.

Evidence: Housing and Urban Development and Labor statistics. Census data. IG report 3-05.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has specific long term goals focusing on outcomes that reflect the purpose of the program. The HUBZone Program supports the Agency's Strategic Goal 2, Increase small business success by bridging competitive opportunity gaps facing entrepreneurs, and Long Term Objective 2.3, Significantly increase successful small business ownership within segments of society. In addition, the agency measures job creation under the program.

Evidence: Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan; U.S. Small Business Administration Performance and Accountability Report, FY2004; and U.S. Small Business Administration Scorecard.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The HUBZone Program has Strategic Goal 2., 'Increase small business success by bridging competitive opportunity gaps (COG's) facing entrepreneurs,' and Long Term Objectives 2.2 and 2.3, 'Maximize the sustainability and growth of existing small businesses (ESB) assisted by SBA,' and 'Significantly increase successful small business ownership within segments of society facing competitive opportunity gaps,' respectively. While these Outcome Measures have been established, they have not yet been base lined. In order to create necessary baselines and measure program outcomes, SBA has developed a tiered evaluation strategy. In 2005, the agency will survey the program participants to assess impact. Further, under the reauthorization of December 8, 2004, the agency must report on the HUBZone Program's impact in 2008.

Evidence: U.S. Small Business Administration Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan; U.S. Small Business Administration Performance and Accountability Report, FY2004; 5-year Strategic Plan; and U.S. Small Business Administration Scorecard

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has specific annual performance measures that demonstrate progress toward achieving long term goals. The program has a limited number of output, intermediate outcome, and outcome measures. Output objectives are as follows: (1) Program examinations completed; (2) HUBZone small business concerns certified annually; and (3) Program recertification actions. The program has the following intermediate outcome measures: (1) Capital investment; (2) Jobs created; (3) Increase in dollar value of contract awards to HUBZone-certified firms; (4) Existing small businesses assisted; and (5) Percentage customer satisfaction. The HUBZone Program intends to use the results of the Decennial Census of 2010, and that of 2000, to analyze the economic impact of the program over the decade since program inception. This assessment will be done on a community-by-community basis. The data and analysis will illuminate assessment of progress in attaining HUBZone Program long term objectives

Evidence: Evidence: U.S. Small Business Administration Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan; U.S. Small Business Administration Performance and Accountability Report, FY2004; and U.S. Small Business Administration Scorecard

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has baselines and ambitious targets for its annual measures. The program has the following output, intermediate outcome, and outcome measures: (1) Program examinations completed; (2) HUBZone small business concerns certified annually; (3) Program recertification actions; (4) Capital investment; (5) Jobs created; (6) Increase in dollar value of contract awards to HUBZone-certified firms; (7) Existing small businesses assisted; and (8) Percentage customer satisfaction. Quantifiable, realistic, but challenging baselines have been established for each of these measures except 'Increase in dollar value of contract awards to HUBZone-certified firms,' and 'Increase in capital investment in HUBZone firms,' 'and ' Percentage customer satisfaction.'' Appropriate baselines for these intermediate outcome and outcome measures will be established during FY 2005, per performance plan.

Evidence: U.S. Small Business Administration Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan; U.S. Small Business Administration Performance and Accountability Report, FY2004; and U.S. Small Business Administration Scorecard.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Acquisition agencies commit to attainment of the statutory goal of awarding 3% of the dollar value of Federal contracts to HUBZone-certified firms, annually. These commitments are critical to achievement of the strategic goals and long term objectives of the HUBZone Program. They bear directly upon accomplishment of the following intermediate outcome measures and outcome measures: (1) Capital investment; (2) Jobs created; (3) Increase in dollar value of contract awards to HUBZone-certified firms; (4) Existing small businesses assisted; and (5) Percentage customer satisfaction. By regulation, agencies are required to report performance against goals through the Federal Procurement Data System-Next Generation (FPDS-NG). This enables evaluation of acquisition agencies' performance in meeting HUBZone Program contracting targets. Determination of HUBZone Program effectiveness, impact, and success ultimately hinge upon the relevance, reliability, and timeliness of contracting information derived from FPDS-NG, and the general perception of credibility of such information.

Evidence: Commitments are documented throughout the goaling process. SBA conducts mid-year assessments of accomplishments against contracting goals. If they fail to achieve goals, agencies must provide corrective action plans. This information is included in the Annual Report to the president

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The HUBZone program has had two major independent evaluations. One review was completed by the Government Accountability Office (GAO) in October of 2002. The second, an audit by the Office of the Inspector General, was completed in January 2003. Both of these evaluations focused on program management rather than program effectiveness or impact. While no independent evaluations have been conducted to assess HUBZone effectiveness, impact, or relevance, the Agency is taking a number of steps to evaluate the program, and identify opportunities for improvement. First, beginning in FY 2005, the HUBZone Program will capture program impact and evaluation information. Using an Internet-based survey instrument, firms will be surveyed regarding contract opportunities secured under the HUBZone Program, and attendant capital investment, job growth, and employment of HUBZone residents. Results will be used to improve the program. Second, the Agency has hired the Urban Institute to assess HUBZone Program impact. Third, the agency is required by statute to prepare a study of the HUBZone Program's impact during FY 2008. Fourth, the Agency intends to use the results of the Decennial Census of 2010, and that of 2000, to analyze the economic impact of the HUBZone Program. The Analysis will be done on a community-by-community basis, and results will be used to gauge program effectiveness. Together, these measures will ensure rigorous systematic evaluation of the HUBZone Program as a basis for programmatic improvement.

Evidence: Report GAO-02-57, and Inspector General Report 03-05.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget requests are tied to accomplishment of annual and long term performance goals. Since program inception, annual funding has been closely aligned with programmatic function. Specifically, HUBZone Program budget requests support the Agency's Strategic Goal 2, Increase small business success by bridging competitive opportunity gaps facing entrepreneurs, and Long Term Objective 2.3, Significantly increase successful small business ownership within segments of society. The Agency has engaged in full allocation of costs in formulation of its budget estimates for several years. Resources consumed by the HUBZone Program aggregated $4.8 million, $6.5 million, $5.6 million, and $6.8 million for FY's 2001, 2002, 2003, and 2004, respectively.

Evidence: U.S. Small Business Administration Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan; U.S. Small Business Administration Performance and Accountability Report, FY2004; and U.S. Small Business Administration Scorecard.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: No strategic planning deficiencies have been recently identified regarding the HUBZone Program through internal or external review or evaluations. However, the program has developed its strategic plans within budgetary levels to maintain and increase program integrity as the portfolio of HUBZone-certified firms has grown.

Evidence: Reference internal memoranda regarding program evaluations.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The HUBZone Program collects internal and external performance information and uses it to manage and improve program performance. (1) The Agency uses FPDS-NG data for determination of compliance with government-wide contracting goals for the HUBZone Program. Unfortunately, there have been problems with the reliability of this data and the General Services Administration is working to address system deficiencies. (2) The HUBZone System provides real time analysis of certification and program examination activities against performance targets, used in day-to-day management of the program. (3) During FY 2005, the Agency will begin collecting longitudinal evaluation information from firms certified under the program, including changes in revenue, total employment and employment of HUBZone residents, and capital investment. This information will be used to manage the program and adjust program design, policies, and procedures to ensure that it actually supports increased employment, capital investment, and economic development. (4) The HUBZone Program has contracted with its geo coding/mapping vendor to provide specific socio-economic data for qualified HUBZone areas, including median household income, average educational level, residential/commercial real estate values, occupancy rates, and other relevant information to baseline economic development from the 2000 Decennial Census to the 2010 Decennial Census. This comparative analysis will allow the office to measure the economic development impact of the HUBZone Program in these communities

Evidence: The Agency issues a Report to the President on government-wide procurement goal attainment. Reports on certification and program examination activity are dynamically available from the HUBZone System.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Program managers and staff are held accountable for cost, schedule, and performance results through establishment of annual performance work plans. Individual performance work plans reflect program office performance measures. Performance measures are directly linked to the Agency's strategic goals and long term objectives. Accomplishment against intermediate outcome and outcome measures hinges upon attainment of procurement preference goals for the HUBZone Program. The Agency executes an annual goaling process and acquisition agencies are held accountable through evaluation of contracting information from FPDS-NG.

Evidence: Performance work plans for staff of the Office of HUBZone Program and U.S. Small Business Administration Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan. Performance and Accountability Report Fiscal Year 2004.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Obligation of funding is based on significant advanced planning, arising out of budget formulation and operational planning processes. Funding is obligated and expended in a timely fashion, and used exclusively for execution of HUBZone Program statutory mandates. Routinely, Program managers work with staff of the Office of the Chief Financial Officer to ensure that actual expenditures against obligations are reported, compared with operating plans to determine compliance with intended use, and adjustments made, as necessary. Particular care is taken by program managers to minimize or preclude an unobligated balance at year-end.

Evidence: U.S. Small Business Administration Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan; Annual Operating Budget for the HUBZone Program.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: SBA utilizes an activity-based costing system to measure cost efficiency on a per unit basis. Since inception, the HUBZone Program has sought to use technology to the maximum extent practicable in executing its business processes. It has done this substantially through phased development and deployment of the HUBZone System. During FY 2004, the program deployed a fully electronic, Internet-based program examination system that provides for routing of work between district offices and headquarters, and incorporates substantial decision support. In so doing, it minimized the compensation and benefits costs associated with intense scrutiny of 5% of its portfolio each year. In July 2005, the system will be expanded to include program recertification. During FY 2006, the Agency will complete the system by deploying a module for adjudication of eligibility appeals and protests. Similarly, the Agency has 'outsourced' significant portions of HUBZone Program administrative and analytical requirements. By investing in technology and outsourcing support requirements, the Agency has been able to minimize fixed staffing cost for a major procurement preference program. That is, an average of ten full time equivalent staff and three contract staff manage a portfolio of more than 13,000 firms with contracting activity aggregating $3.4 billion (FY 2003).

Evidence: U.S. Small Business Administration Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan; Annual Operating Budget for the HUBZone Program; Contract for Development and Maintenance of the HUBZone System; Contract for Provision of Support Services to the HUBZone Program; Agency Scorecard

Does the program collaborate and coordinate effectively with related programs?

Explanation: Since inception, the HUBZone Program has collaborated extensively and coordinated effectively with the acquisition and economic development communities. The HUBZone Program routinely works with state and local economic entities to encourage use of the program as an economic development tool, and to encourage development of local initiatives that leverage the program. During the balance of FY 2005 and throughout FY 2006, the HUBZone Program will launch an initiative to work directly with state and local economic development agencies across the country. Under the initiative, the Agency will target economically distressed communities in each state. It will then work with relevant authorities to integrate the HUBZone Program into their economic development strategic planning. The Agency has already developed and pilot tested a model customizable package for distribution to states and economically distressed jurisdictions therein. It is expected that much of the coordination for this initiative will be executed by economic development specialists in the Agency's district offices. The intent of this effort is to amplify the impact of HUBZone procurement preferences in local communities, through programs that complement or emulate the program. The program routinely coordinates its activities and initiatives with other programs internal to SBA. For example, because they are located in areas of high unemployment and/or low income, essentially all firms certified under the HUBZone Program are eligible for receipt of management and technical assistance under Section 7(j) of the Small Business Act. Therefore the HUBZone Program works with the 8(a) Program to ensure availability of 7(j) assistance appropriate to HUBZone firms. Similarly, the program collaborates with the Prime Contracting Program on conduct of Business Matchmaking activities that will benefit HUBZone firms.

Evidence: U.S. Small Business Administration Scorecards for FY's 2003-2005.

Does the program use strong financial management practices?

Explanation: The program works closely and continuously with the Office of the Chief Financial Officer to ensure application of best financial practices. It engages in budget formulation and operational planning processes that link resource requests to the Agency's strategic goals and long term objectives, and specific measurable outputs and outcomes. All obligations are made in accordance with statutory authority, and expenditure of funds against obligations is authorized and carefully monitored. There are no audit material weaknesses associated with the HUBZone Program, nor are there any deviations from the requirements of proper financial management. As elsewhere discussed, performance planning and budget formulation are fully integrated. Obligations and expenditures are thoroughly reviewed at least monthly. This review encompasses the appropriateness of payments against obligations to preclude improper payments.

Evidence: U.S. Small Business Administration Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan; Annual Operating Budget for the HUBZone Program; Performance and Accountability Report, FY 2004; Agency Scorecard.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The HUBZone Program must maintain strong program integrity in terms of firms' continuing eligibility, as its portfolio of certified firms grows. This deficiency was identified in the Inspector General's Report 03-05. The program has addressed this concern by implementing an internet-based program examination/compliance review process and plans to review 5% of its portfolio annually. The Associate Administrator for HUBZone Program and program managers monitor performance against established measures on a continuous basis. Plans are developed to address any management issues identified through such monitoring, and are included in the Agency's Scorecard for the HUBZone Program, and in the personal business commitments (performance work plans) of involved managers and staff.

Evidence: Response to the referenced audit. Documentation for the HUBZone System Program Examination Module. Agency Scorecard for the HUBZone Program FY's 2003-2005. Personal Business Commitments

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Limited data is available to assess whether the HUBZone is achieving long-term performance goals. However, the Agency is taking a number of steps to determine program progress. Beginning in FY 2005, the HUBZone Program will capture program impact and evaluation information. Using an Internet-based survey instrument, firms will be surveyed regarding contract opportunities secured under the HUBZone Program, and attendant capital investment, job growth, and employment of HUBZone residents. Results will be used to improve the program. As noted earlier, the Agency's overall program evaluation activities, including, but not limited to, an effort to be conducted by the Urban Institute, will assess HUBZone Program impact, beginning as early as FY 2006. Perhaps more importantly the Agency will use the results of the Decennial Census of 2010, and that of 2000, to analyze the economic impact of the HUBZone Program. Analysis will be done on a community-by-community basis, and results will be used to gauge program effectiveness. Finally, as elsewhere described, recently enacted legislation requires that the agency prepare a study of the HUBZone Program's impact during FY 2008. These measures will ensure rigorous systematic evaluation of the HUBZone Program as a basis for programmatic improvement

Evidence: Small Business Reauthorization and Manufacturing Assistance Act of 2004, and U.S. Small Business Administration Congressional Submission Fiscal Year 2006.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The HUBZone Program has exceeded its targets for certification of firms and conduct of program examinations. Further, the program expects to meet or exceed its newly established performance target for recertification of firms. However, the program has not achieved its targets for award of contracts to HUBZone firms. Achievement of these targets bear directly upon accomplishment of the following intermediate outcome measures and outcome measures: (1) Capital investment; (2) Jobs created; (3) Increase in dollar value of contract awards to HUBZone-certified firms; (4) Existing small businesses assisted; and (5) Percentage customer satisfaction. SBA and acquisition agencies negotiate goals for award of contracts to HUBZone-certified firms and are committed to fulfillment of those commitments.

Evidence: U.S. Small Business Administration Congressional Submission Fiscal Year 2004-2006, Budget Request & Performance Plan.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has operated within budget and accomplished its goals. The cost per unit for the HUBZone Program averaged $2,611 from FY 2001 through FY 2004, fluctuating from $2,347 in FY 2002 to $2,975 in FY 2004. Over the same period, the cost per unit for the 8(a) Program averaged $4,975, fluctuating from $4,425 in FY 2003 to $6,082 in FY 2002. For FY 2004, 8(a) per unit cost was $4,627. Cost per unit for the HUBZone Program is based on the number of firms certified annually. The FY 2004 cost increased due to the addition of program examination in that year. Excluding this new function, unit cost for FY 2004 would fall from $2,975 to $2,523. In terms program efficiency, it is notable that the value of contracts awarded to HUBZone-certified has risen consistently each year. For example, from FY 2002 to FY 2003, the value of contracts awarded to HUBZone-certified firms essentially doubled, rising from $1.7 billion to $3.4 billion. As significantly, the yield in value of contracts awarded to HUBZone-certified firms to annual investment in the HUBZone Program increased from 284:1 in FY 2002 to 606:1 in FY 2003. More importantly, in terms of intermediate outcomes, assuming that approximately $133,500 of contract value supports one, job the number of jobs held by HUBZone residents supported by awards to HUBZone-certified firms increased from 9,000 in FY 2002, to 17,900 in FY 2003. Further, program cost per HUBZone job supported declined from $723 for FY 2002, to $315 in FY 2003. In the context of level funding, and level-to-declining staffing since inception, as noted elsewhere, the HUBZone Program has outsourced substantial portions of its case analytical certification activities and all systems development requirements. This strategy has allowed the program to retain maximum flexibility, shifting application of staffing and financial resources as dictated by program development and requirements. The program also has made substantive investments in technology. By automating all substantive business processes, the program has been able to minimize its staffing requirements. These efforts have enabled the HUBZone Program to provide exceptionally responsive service to the public at the least possible cost. They have also enabled the HUBZone Program to maintain and increase program integrity and oversight, and to focus more and more on systematically increasing contracting opportunities for HUBZone-certified firms.

Evidence: U.S. Small Business Administration Congressional Submission Fiscal Year 2004-2006.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Until further evaluations are undertaken, it will be difficult to ascertain how the HUBZone program compares with other procurement preference programs in terms of outcomes. As noted elsewhere, while different from the HUBZone Program in purpose and intent, the 8(a) and SDB Programs are, to varying degrees, peer procurement preference programs to it. The HUBZone Program attained a critical mass of certified vendors within a three-year period, while the 8(a) Program required twenty-five years to attain a similarly sized portfolio. Further, the value of contract awards under the HUBZone Program increased by 416% from FY 2000 to FY 2003. During the same period, the value of SDB and 8(a) contract awards increased 29% and 75%, respectively. All of the HUBZone Program's business processes are fully automated, while those of virtually all other preference programs are substantially manual. For this reason, HUBZone Program's unit costs are more moderate in comparison with other programs, and in the context of program impact on distressed communities. The cost per unit for the HUBZone Program for Competitive Opportunity Gap firms assisted in FY 2004 was $2,975, while the analogous cost per unit for the 8(a) Program was $4,627. Thus, cost per unit for the HUBZone Program was approximately 36% lower than that of the 8(a) Program.

Evidence: Records for comparable certification program appear in publication "State of Small Business" issued by SBA's Office of Advocacy; U.S. Small Business Administration Congressional Submission Fiscal Year 2006, Budget Request & Performance Plan; Annual Operating Budget for the HUBZone Program.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: To date an evaluation of HUBZone outcomes is in process and due to be completed in FY 2005. The Agency has taken a number of steps to systematically implement this evaluation of the HUBZone Program and to identify opportunities for improvement. In FY 2005, the Agency began capturing program impact and evaluation information. Using an Internet-based survey instrument, firms were surveyed regarding contract opportunities secured under the HUBZone Program, and attendant capital investment, job growth, and employment of HUBZone residents. The Agency deployed the data collection tool on June 16, 2005. Data capture and data assessment will be complete by September 30, 2005. Preliminary data as of August 1, 2005 from a total of 3,360 respondents shows the following results: Total number of employees hired as a result of the firm's HUBZone certification = 10,836; Total number of employees who reside in a HUBZone who were hired as a result of the firm's HUBZone certification = 6,654; Total capital investment increase in the firm as a result of the firm's HUBZone certification = $512,908,139.98. Recently enacted legislation requires that the agency prepare a study of the HUBZone Program's impact during FY 2008. Further, the Agency will use the results of the Decennial Census of 2010, and that of 2000, to analyze the economic impact of the HUBZone Program. The analysis will be done on a community-by-community basis, and results will be used to gauge program effectiveness. These measures will ensure rigorous systematic evaluation of the HUBZone Program as a basis for programmatic improvement

Evidence: Small Business Reauthorization and Manufacturing Assistance Act of 2004, and U.S. Small Business Administration Congressional Submission Fiscal Year 2006; General Accounting Office Report GAO-02-57 "HUBZone Program Suffers from Reporting and Implementation Difficulties;" OFPP policy guidance.