Detailed Information on the
Rural Business-Cooperative Service Value-Added Producer Grants Assessment

Assess opportunities for reducing the program burden on applicants and Rural Development.

Continually re-assess existing performance measures and evaluate potential new measures. Replace existing measure(s) when a new measure is discovered that will better reflect program objectives and performance.

Increase targeting of program to emerging markets. Continue to assess the focus of the program on small and medium-sized producers.

Measure: The percentage of businesses assisted that are still operational 3 years after the project is completed.

Explanation:The program provides grants to independent agricultural producers to market value-added agricultural products. The sustainability of businesses receiving Federal assistance is critical to measuring the success of the program. The ultimate target has been set at 60% of business assisted that are still operational 3 years after the project is completed. This target is consistent with research on the sustainability of new businesses. However, data indicates that the Grantees are performing below the national average. Possible explanations for this performance may include funding riskier ventures, failing to provide sufficient assistance to grantees during the difficult start-up time of their ventures, and inadequate selection procedures. Since the 2001 and 2002 grants were made, the Agency has implemented a regulation, and improved its selection criteria. Future steps include providing more training to State Offices on monitoring a grant and identifying early warning signs that a Grantee is experiencing difficulty completing its project as well as solidifying requirements for a feasibility study and business plan to meet Agency approval. FY 2001, the first year of the program, is the base year for this measure. Grants in FY 2001 were awarded in September and were for at least one-year periods of performance. Thus, the data for this measure began to be reported as of September 2004. Only grants awarded through 2002 are being reported at this time because grants awarded from funds appropriated in 2003 were not made until the fall of 2003, meaning that the three-year time period will not occur until the fall of FY 2006.

Measure: The three-year average of the percentage of grants completed within 18 months of obligation date.

Explanation:Given that the grants are designed to be fully utilized within a one-year time period, this measure is critical in determining if personnel are setting up the grants, servicing the grants, and closing out the grants in an expedient manner. A three-year average allows us to set annual targets and then average our annual targets over time to ensure that we continue to improve our servicing of the grant awards. RD believes a three year average is better than a longer-term average, such as five or ten years, because the program is still fairly new, with grants first awarded in 2001. Thus, RD's baseline is an average of 2001 through 2003 awards. RD does not yet have data for 2004 and beyond because the grants have not yet reached the 18 months after obligation point, so a measure incorporating more than 3 years would not be useful at this point. The ultimate target is to have 95% of grants completed within 18 months of obligation. However, yearly increments have been established to reach that target. The increments are small because the measure is a three-year average, so an increase in the percentage in one year will not be fully seen in the three-year average. Also, the target for the 2002-2004 year is smaller than future years because RD has little control over grants that have already been completed in from the 2002 and 2003 years. Efforts will focus on 2004 and future grants to improve the timeliness of disbursements.

Measure: The number of jobs created by businesses assisted.

Explanation:Rural Development has a goal of improving quality of life in rural areas, in part through job creation. This measure supports Rural Development goals. The baseline has been established based on data reported by 2004 Grantees who have completed their projects as of the 2nd quarter of FY 2006. FY 2005 Grantees are scheduled to complete their grants in the fall/winter of FY 2007, so no data is available at this time, but targets for future years are based on what has been reported by 2004 Grantees and what funds have already been appropriated for each year or what is proposed in the President's budget.

Measure: Maintain or increase the business revenue per project.

Explanation:The statute states the purpose of the program is to increase revenues to independent agricultural producers. This measure is critical to determining if the program is achieving its statutory purpose. Targets and baseline established following examination of early progress reports. Changes in grant cap limits impact future targets. In previous years, the portfolio consisted of ethanol and other producers that have entered a fast growing and highly sustainable market. These producers carry little risk and can easily obtain support in the private sector, explaining the high levels of business revenue. As the portfolio changes to support producers that have harder times receiving funding from other sources, carry a greater risk, the level of difficulty in maintaining the Target becomes increasingly difficult.

Measure: The expansion of customer base per project.

Explanation:The statute states the purpose of the program is to expand the customer base of independent agricultural producers. This measure is critical to determining if the program is achieving its statutory purpose. The baseline year is 2004. Targets have been established following examination of early progress reports. In previous years, the portfolio included some projects in highly sustainable markets. These projects carry less risk and can more easily obtain support in the private sector, explaining the high levels of business revenue. RD is currently considering increasing targeting of grants to emerging markets, with less emphasis on awarding grants for projects in well-developed industries. This targeting would likely increase the difficulty in maintaining the baseline value.

Measure: Cost per application processed.

Explanation:Given annual increases in cost-of-living adjustments for Federal and non-Federal personnel, it is the goal of the program to maintain current efficiency levels to minimize the increase in cost of processing each application. Substantial cost savings were realized in FY 2005 because of a conversion from outsourcing the processing to performing the work in-house. The baseline year is 2004. However, changes in the way applications are processed in FY 2006 have driven processing costs up due to increased burden on RD staff. Program managers will evaluate the benefit of the processing changes as compared to the costs incurred by the changes.

Is the program purpose clear?

Explanation: The Value-added Producer Grant program awards competitive grants to Independent Producers, Agriculture Producer Groups, Farmer or Rancher Cooperatives, and Majority-Controlled Producer-Based Business Ventures. These entities must be composed of or represent producers of agricultural commodities. Grants may be used for planning activities, such as feasibility studies and business plans or for working capital expenses associated with marketing a Value-Added agricultural product. The purpose of this program is to increase revenue accruing to producers and to assist producers with expanding the customer base for Value-Added products.

Evidence: 7 U.S.C. 1621 note and 7 CFR Part 4284 (http://www.gpoaccess.gov/cfr/index.html). The program statute is very specific about eligibility and use of funds. It clearly indicates the program objectives when it states, "??[A]s a result of the change in physical state or the manner in which the agricultural commodity or product was produced or segregated??(i) the customer base for the agricultural commodity or product has been expanded; and (ii) a greater portion of the revenue derived from the marketing, processing, or physical segregation of the agricultural commodity or product is available to the producer of the commodity or product." 7 CFR Part 4284 implements the authorizing legislation and includes definitions of the eligible entities and descriptions of the eligible purposes of the grants. It also defines Value-Added in §4284.3 as, "The incremental value that is realized by the producer from an agricultural commodity or product as the result of a change in its physical state, differentiated production or marketing, as demonstrated in a business plan, or product segregation. Also, the economic benefit realized from the production of farm or ranch based renewable energy."

Does the program address a specific and existing problem, interest, or need?

Explanation: The farmers share of the U.S. retail food market basket has declined from a 1973 high of 44 percent to just 21 percent in 2001. The statutory objectives of the VAPG program are to increase agricultural producers' revenue and customer base through marketing Value-Added products, thus assisting producers with gaining a greater share of the value added to commodities as those commodities are processed and sold at wholesale or retail levels. Grants may be used for planning activities, such as feasibility studies and business plans or for working capital expenses associated with marketing a Value-Added agricultural product. Grants are awarded through a competitive process that evaluates the project's potential at meeting these objectives.

Evidence: ERS Market Basket Index (http://www.ers.usda.gov/Briefing/FoodPriceSpreads/basket/table1.htm). "Retail Food Prices Increase Slightly." Pork News Source, July 11, 2005. 7 U.S.C. 1621 note describes the program's objectives. 7 CFR 4284.913 (http://www.gpoaccess.gov/cfr/index.html) outlines the evaluation criteria for the grant applications. For planning grants, the first criterion states, "??Points will be awarded based on the greatest expansion of markets and increased returns to producers??" For working capital grants, the second criterion states, "Proposals that demonstrate strong growth in a market or customer base and greater Value-Added revenue accruing to producer-owners will receive more points than those that demonstrate less growth in markets and realized Value-Added returns."

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The VAPG program provides funding for value added crops. Though it is the biggest federal grant program specific to value added products, many of the currently funded commodities can obtain funding through other federal sources. Renewable energy, i.e. ethanol and biodesel commodities, can be funded through other rural development programs such as the Renewable Energy Program, Rural Business Enterprise Grants, and the Business and Industry Guaranteed Loan program (B&I). Additionally, Rural Development allows a producer to leverage grant funding using subsidized loans from elsewhere in USDA. To obtain a value added grant, a recipient must provide matching funds. Those matching funds could come from a loan program such as B&I. A report run on April 28, 2006 from the GLS system indicates that 6% of VAPG recipients since 2001 have also received funding from other RD programs. Additionally there are numerous state programs, such as Illinois that provided a $2 million dollar grant for an ethanol cooperative, that provide funding for value added crops. These grant programs primarily occur in midwestern states and can have value added grant caps at as low as $5,000. To improve, and decrease their redundancy, the Agency has entered into a Grant Agreement with the University of Missouri that will, in part, assess value-added programs other than the VAPG and Agriculture Innovation Center programs. Thus, the Agency can update its knowledge of other value-added programs and deversify its porfolio to compensate for the redundancy.

Evidence: VAPG regulations, see the following website: http://www.rurdev.usda.gov/rbs/coops/regs.htm Hancock, Samuel J. and William A. Thomas. A Survey of State Value-Added Grant Programs. 2002, University of Georgia. It can be viewed online at: www.agecon.uga.edu/~caed/valueHancock&Thomas.pdf. A Grant Agreement with University of Missouri was entered into in April 2005. All deliverables are expected by April 1, 2007. GLS report dated April 28, 2006 shows grantees receiving assistance from the VAPG program and other RD programs.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program is administered by USDA Rural Development with oversight at the National Office level and grant servicing done at the State and area office level. Federal and program regulations in conjunction with annual application solicitations in the Federal Register set forth eligibility for the program as well as application processing and grant servicing procedures. All grantees must meet the terms and conditions for their awards as stated in a Letter of Conditions and the Grant Agreement. The Grant Agreement specifies the projects will be monitored to assure compliance with applicable regulations. The VAPG program also uses matching funds, in accordance with the statutory and regulatory requirements, as a leveraging mechanism to help achieve the program's intended purpose. Any suspected erroneous payments are first investigated by RD field office staff and then if warranted, the matter is referred to the Office of the General Counsel for further guidance. 7 CFR 1951 and the associated RD Instruction apply in cases where unauthorized assistance is discovered. A three phase OIG audit is currently on-going with phase 1 completed, phase 2 near completion, and phase 3 scheduled to begin once phase 2 is complete. The VAPG program is also is scheduled for a Management Control Review in FY 2008. The MCR assesses whether the program is administered in compliance with applicable regulations. The VAPG program is also scheduled to become part of the State Internal Review process in FY 2007. The SIR assesses whether the field offices are in compliance with applicable regulations. The Agency is not aware of any cost-benefit studies that evaluate the mechanism of this program in comparison with using another mechanism.

Evidence: 7 CFR Parts 11, 1951, 3015, 3017, 3018, 3019, 3052, and 4284, subparts A and J govern the program's administration. All CFR citations can be viewed at: http://www.gpoaccess.gov/cfr/index.html. OMB Circular A-122 (http://www.whitehouse.gov/omb/circulars/a122/a122_2004.pdf) is used as a Cost Principle for non-profit Grantees and the Federal Acquisition Regulation (http://www.arnet.gov/far/) is used as a Cost Principle for commercial organizations. The Letter of Conditions (LOC) and Grant Agreement templates may be customized to address high-risk Grantees. The LOC sets forth the terms and conditions under which the grant will be made. The Grant Agreement is a comprehensive document incorporating all applicable regulations and reminding Grantees of their obligations with respect to especially important grant requirements, including financial management, procurement standards, reporting, site visits, records, and enforcement of the grant requirements.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: All applications are screened at both the state and national levels to identify ineligible applicants, in accordance with published procedures. Once initial funding selections have been made, successful applicants are once again screened for eligibility, and continue to be screened for eligibility throughout the grant period by reviewing required semi-annual performance and financial reports, by conducting site visits, and by reviewing or conducting audits, as required by 7 CFR 3052, OMB Circular A-133, and the Grant Agreement. After the Administrator of the Rural Business-Cooperative Service has made his/her funding selections based on the procedures published in the annual Notice of Solicitation of Applications (NOSA), the National Office moves money appropriated to the program to State Office accounts. The State Offices are responsible for obligating funds to the approved grantees. No funds are appropriated for administrative costs of the program. All appropriated funds are awarded to grantees. If any funds are unused by a grantee, they are carried over to the next fiscal year, if possible, or revert to the Treasury, if they cannot be carried over. The Agency uses part of its S&E budget to pay the salaries of the employees working on the program, but no other funds are expended. Because grants are awarded directly to the intended beneficiaries, program benefits reach the agricultural producers without first going through an intermediary. Thus, grantees can utilize the grants directly and no additional costs associated with pass-through grants are realized. The program is targeted toward the intended beneficiaries by advertising on a program website, in the Federal Register, in the CFDA, and on Grants.gov. RD field offices also conduct outreach by doing presentations about the program and working with local papers to announce funding availability. Both the National Office and field offices work with industry associations, such as the National Corn Growers Association, to make sure that program questions are answered. Activities that would have occurred without the program are not subsidized because the program regulation, 7 CFR 4284.10, restricts the use of grant funds to activities that are new or expanded beyond current activities. No funds may be used to duplicate current services or replace or substitute support previously provided.

Evidence: 7 CFR Parts 1951, 3052, and 4284 (http://www.gpoaccess.gov/cfr/index.html); OMB Circular A-133 (http://www.whitehouse.gov/omb/circulars/a133/a133.html); FY 2006 Notice of Solicitation of Applications (NOSA) (http://a257.g.akamaitech.net/7/257/2422/01jan20051800/edocket.access.gpo.gov/2005/pdf/E5-7596.pdf); VAPG Grant Agreement; OIG Audit Report No. 34601-3-KC (http://www.usda.gov/oig/webdocs/34601-3-KC.pdf); program website at: http://www.rurdev.usda.gov/rbs/coops/vadg.htm. 7 CFR 1951 provides guidance to the field office regarding servicing of the grants. 7 CFR 3052 and OMB Circular A-133 provide guidance on audits for the grants. 7 CFR 4284 and the annual NOSA provide specific eligibility requirements for applicants, proposed projects, and use of funds. The Grant Agreement provides guidance to the Grantee on maintaining compliance with applicable regulations. The program website provides assistance to applicants by providing contact information, a link to any current solicitation, a list of previous recipients, a link to the program regulation, an Application Guide with a completeness checklist, an Application Template, and a Frequently Asked Questions list. A three phase OIG audit is currently on-going with phase 1 completed, phase 2 near completion, and phase 3 scheduled to begin once phase 2 is complete. The phase 1 audit report indicated that National Office internal controls needed improvement. However, since that report was released in April 2004, the National Office has implemented some of the OIG recommendations and is negotiating with OIG on the other recommendations to ensure proper program controls. Also, at the time the audit was completed, the program did not yet have a program regulation, which it now has. This regulation, 7 CFR 4284, explicitly defines the eligibility requirements for the program and the processing and servicing procedures.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has two long-term performance measures, one focusing on an outcome, and the other focusing on efficiency. The outcome measure is the percentage of businesses assisted that are still operational three years after the grant is completed. One of the goals of the program is to assist producers with achieving sustainable businesses, thus reducing their dependence on non-farm income and increasing their revenues from marketing value-added agricultural products. The Agency selected the three year period because research indicates that after two years, business attrition seems to stabilize. Also, the Agency's record-keeping requirements indicate that records should be disposed three years after the assistance is completed, thus adding additional time to this measure and remaining in compliance with record retention instructions is not possible. The long-term efficiency measure is a three year average of the percentage of grants that are completed within 18 months of obligation. It is important that grant funds are expended in a reasonable time frame, so that performance of the grants can be tied with appropriation and reported in a timely fashion, thus allowing program decision-makers can make appropriate budget decisions.

Evidence: These performance measures have been entered into the Department's performance measure tracking system, and initial data to establish the baselines and targets is being collected and analyzed. For the outcome measure, the base year will be 2001. Preliminary reports indicate approximately 70% of the businesses assisted are still in operation three years after the grant is completed. For the efficiency measure, the base year will actually be a three year average of 2001-2003. Data has already been collected for these years, although not all grants awarded have been completed. Preliminary data indicates the average for all closed grants is 501 days between obligation and the date of the last reimbursement. Further analysis needs to be completed, and an initial report on these performance measures is expected by June 2006. The Agency currently collects data for the outcome measure through RD field office reports and through GLS. Summary reports for the measures are maintained internally. RD Instruction 2033-A contains record-keeping requirements and can be viewed at: http://www.rurdev.usda.gov/regs/regs_toc.html#2033. Headd, Brian. "Redefining Business Success: Distinguishing Between Closure and Failure." Small Business Economics 21: 51-61, 2003. This article discusses a couple of studies that have been completed on business closures.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The ultimate target has been set at 60% of business assisted that are still operational 3 years after the project is completed. This target is consistent with research on the sustainability of new businesses. However, data indicates that the Grantees are performing below the national average. Possible explanations for this performance may include funding riskier ventures, failing to provide sufficient assistance to grantees during the difficult start-up time of their ventures, and inadequate selection procedures. Since the 2001 and 2002 grants were made, the Agency has implemented a regulation, and improved its selection criteria. Future steps include providing more training to State Offices on monitoring a grant and identifying early warning signs that a Grantee is experiencing difficulty completing its project as well as solidifying requirements for a feasibility study and business plan to meet Agency approval. While many of the businesses assisted are new businesses, some businesses have been in existence for several or more years. In either case, it is the goal of the program to assist businesses that propose projects with a high likelihood of success while recognizing the realities of the business world - that many new businesses fail. The ultimate goal for measuring the three-year average of the percentage of grants completed within 18 months of the obligation date is 95%. The Agency cannot strive for 100% because it must recognize that not all grants will go as expected, and a few grantees may need extensions beyond a short time period to complete their projects.

Evidence: Performance measures have been entered into the Department's performance measure tracking system, MITS. Headd, Brian. Redefining Business Success: Distinguishing Between Closure and Failure. Small Business Economics 21: 51-61, 2003. Research indicates that new businesses open after two years ranges between 66 and 76 percent, new businesses open after four years ranges between 47 and 50 percent, and new businesses open after six years ranges between 38 and 40 percent. GLS Report dated April 28, 2006 contains the preliminary information on percentage of grants completed within 18 months after obligation. The percentages by year have not yet been calculated, but an average of all grants completed as of April 28, 2006 is 501 days, less than 18 months.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has three annual output measures, and one annual efficiency measure. The three annual output measures are the number of jobs created by the businesses assisted, the increase in producer revenue per grant, and the expansion of customer base per grant. The efficiency measure is cost per application. The three annual output measures are based on the statutory objectives of the program and they support the long-term measure of the percentage of businesses still in operation three years after the grant is completed. The three output measures are necessary in reaching the long term goals of the program. Additional outcome measures that support the long term performance measure would have been duplicative and redundant in capturing program activities.

Evidence: The performance measures have been entered into the Department's performance measure tracking system, and initial data to establish the baselines and refine the targets is being collected. The base year for the jobs, revenue, and customer base measures is 2004. The grants awarded in 2004 are being completed largely during the December 2005 through June 2006 time frame. Initial data has been collected for those grants completed by December 2005, and preliminary analysis indicates the number of jobs created is 591, the increase in revenue per grant is $1.9 million, and the increase in expansion of customer base per grant is 107. An initial report on the performance measures is expected by June 2006. The cost per application measure also has as its base year 2004. The cost has already been calculated for 2004 at $1,564 per application, and the 2005 cost will be calculated by June 2006, but is currently estimated at $930 per application.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has set baselines and targets for its annual measures. The baseline year for these measures is 2004, the first year the measures were incorporated into the Grant Agreements. The baseline for number of jobs created is 363. Targets for future years are set according to funds that have been appropriated or are proposed in the President's budget. These targets are slightly above the formula used by other RD programs, making the targets very ambitious for a program whose secondary, rather than primary, purpose is to create jobs. The baseline for increase in revenue per grant is $66,933, which is less than the average grant size for the base year. The target is to increase this measure so that it exceeds the average grant size, thus ensuring a 1:1 benefit to cost ratio. The target is ambitious because only working capital grants generate an increase in revenue, so any planning grants awarded (usually about half of the grants awarded each year) will not be contributing to the measure. The baseline for the increase in expansion of customer base per grant is 173 with a target set at 175. Maintaining this target is ambitious because the types of businesses awarded grants vary significantly and are not predictable. Also, the program targets smaller businesses, which will likely have fewer customers. Finally, the cost per application baseline is set at $1,564. The target is set at $1,000 per application because RD has internalized procedures that were formerly contracted out. This internalization has led to significant cost savings. Maintaining this target is challenging in light of yearly cost of living adjustments for employees and potential increases in processing burden to the Agency in an attempt to improve customer service.

Evidence: The 2004 and 2005 Grant Agreements identify the jobs, revenue, and customer base measures as measures that Grantees need to report on at the completion of the grant. The 2006 Notice of Solicitation of Applications identifies the same three performance measures that grantees are expected to report on at the completion of the grant. 2004 grants are largely expected to be completed in the December 2005 to June 2006 time frame. Once a significant number of grants have been completed (at least 50%), the baseline can be estimated for 2004. Though RD does not separate its S&E budget out by program, the VAPG program completed a survey, and used contract and FTE hours to calculate the cost per application. Current numbers are being reviewed and verified by workers in the field.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: VAPG partners support the goals of the program by submitting a report when the grant is completed that identifies the jobs created, increase in revenue, and expansion of customer base attributed to the project. This report is required in the Grant Agreement for grants awarded in 2004 and later. Beginning with the 2006 program, applicants are notified of the performance reporting requirements in the Notice of Solicitation of Applications. Grantees are also required to report project progress and financial information on a semi-annual basis. An overdue report may result in suspension of the grant, and even termination in accordance with 7 CFR 3019 and 4284.

Evidence: Grant Agreements specify the frequency and content of performance and financial reports. The 2006 NOSA puts applicants on notice of the program performance measures and requires the applicants report a starting level for the measures that can be compared with the final report. 7 CFR 3019 and 4284 provide instructions on frequency and content of reports as well as enforcement action

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The VAPG program does not have or complete regular independent evaluations. Created in 2001 by the Farm Bill, the USDA OIG is in the process of completing a three phase audit of the program. In phase one, the VAPG program was subject to OIG review of National Office procedures in 2003. The audit report was issued in 2004. In the second phase the OIG is looking at State Office monitoring of existing grants. In phase three, OIG will review applicant eligibility to examine whether the grants awarded were awarded to eligible entities. Though Rural Development reviews the VAPG program annually internally, the OIG audit is the only independent review planned for the near future.

Evidence: The OIG audit report for phase one can be found at: http://www.usda.gov/oig/webdocs/34601-3-KC.pdf.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Due to Rural Development's structure, administrative costs are not incorporated into the program's budget request. Though VAPG performance measures will be reported as of June 2006 and a link between the program's budget and performance will be made, the full costs associated with running the program are not incorporated. Rural Development's administrative expenses are provided in one lump sum, without program specification. Funding provided in a budget request goes directly to the recipient. Though the funding number may reflect annual and long term performance goals in the 2008 budget, the administrative costs of the program will not be linked making it difficult for the agency to explicitly tie its goals to the conglomerated VAPG budget.

Evidence: See President's Budget for FY 2005, 2006 and 2007.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The VAPG program staff has made significant progress in addressing deficiencies. The final rule for the program and associated RD Instructions were published on April 29, 2004 at 7 CFR 4284. In a continuing effort to refine the existing regulation, Cooperative Programs has begun work on revising the regulation to reduce the burden to both applicants and Cooperative Programs while still maintaining proper management controls and remaining compliant with Departmental grant regulations. These refinements address recommendations from OIG's 2003 audit of the program. Program staff and management have also met to outline long-term strategic goals and to set a time frame for achieving those goals. Six performance measures are approved by Cooperative Programs and reporting mechanisms are identified for each measure. These performance measures will help program staff identify progress towards achieving long-term goals. Also, the program is currently scheduled for a Management Control Review (MCR) in FY 2008. MCRs strive to determine whether necessary controls are in place and producing the intended results, comply with applicable laws and regulations, and provide solutions to reduce or eliminate any deficiencies. The recommendations from the MCR will be incorporated into the program when the MCR is completed.

Evidence: USDA MITS, (the Department's performance measure tracking system), National Office communication to State Offices regarding reporting of performance measure data, GLS reporting system, performance reports from State Office, OIG Audit Report No. 34601-3-KC located at http://www.usda.gov/oig/webdocs/34601-3-KC.pdf, RD Instruction 2006-M (http://teamrd.usda.gov/rd/cfo/fmd_home.htm). The long-term strategic goals include: publishing processing and servicing Administrative Notices, so that the field offices will have improved guidance on administering the program; publishing a revised program regulation that reduces the burden for both applicants and the Agency; updating and enhancing the program website to include more information for applicants on applicable regulations and application tips; being prepared to accept applications and announce awards earlier in the fiscal year; to conduct training for field office staff on monitoring grants. The Agency plans to put the draft revised regulation into clearance in Fall 2006 as well as issue guidance to the field offices. An improved web site is expected by the date of the next NOSA, likely in the Fall of 2006. A training request for grants management training was submitted to the RD Training Board in January of 2006, but no approval has been granted at this time.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: For the VAPG program, the only program partners are grant recipients. The Agency does not partner with other agencies, financial institutions, or contractors at this time. With the 2004 grant awards, all Grantees are required to report their general performance, including meeting proposed objectives and the proposed budget on a semi-annual basis. These Grantees are also required to report their performance with respect to increase in revenue, expansion of customer base, and job creation/retention at the completion of the grant. The data reported is verified by the servicing State Office and then reported to the National Office either through GLS or via e-mail. All performance measure reporting is expected to be automated through GLS by September 30, 2006. Unfortunately, collecting additional data once the Grant Agreement is complete is not possible because the Grantee's obligation to the Agency is complete at that time. The Agency has also chosen not to implement a voluntary survey of past Grantees because of the additional burden that would be imposed. Instead, the Agency has chosen measures that can be reported by the Grantee during the grant period and at the completion of the grant or that can be collected by the Agency without involving the Grantee after the grant is completed. A separate Grant Agreement with the University of Missouri is expected to yield useful information regarding the sustainability and success of Grantees from 2001 through 2003. This deliverable is scheduled to be completed in August 2006. Grantees who do not meet the requirements of their Grant Agreements are subject to corrective action, suspension of the grant, and even termination of the grant if reports are not received on time or if performance is sub-optimal and cannot be improved. By incorporating many of the performance measures into the evaluation of applications, the Agency can use reported Grantee performance to adjust its evaluation of applications to achieve program objectives. The program regulation allows the Agency the flexibility to modify the evaluation criteria weights and add additional criteria if necessary to achieve program objectives. The Agency also plans to use the efficiency measure information to determine what level of burden is appropriate for program staff to process applications effectively and is cost effective. Based on the reported cost per application for 2004, the decision was made by Agency management that the cost was too high, and work that was formerly contracted was moved to in-house staff thus saving the Agency an estimated $634 per application. Baseline data is being collected and analyzed for all measures, with a preliminary report scheduled for June 2006. Targets will be set based on the preliminary report and industry research and will be available in June 2006.

Evidence: Development of GLS data entry screens, communication from National Office regarding performance measure reports, Grant Agreements, performance reports, 7 CFR 4284.913.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Administrator of RBS and the Deputy Administrator of Cooperative Programs have their performance evaluated, in part, based on the cost, schedule, and performance of the program. These items are also being integrated into the performance evaluation criteria for other program staff. Grantees are also held accountable for cost, schedule, and performance results. Grantees must expend only the approved amount of grant funds on their projects - no cost overruns are permitted. RD State Offices monitor grantee work plans, cost, and performance on a semi-annual basis through reports and site visits. Deviations from the work plan require approval from the Agency. Performance measures are identified in the Grant Agreement. Grantee performance is monitored by RD field staff during the grant period only. Also, Grantees are paid on a reimbursement basis only, so prior to disbursing the last payment, RD staff ensure that the work done during the grant period conforms to the approved budget and work plan. Any future grant awards are linked to the ability of the Grantee to complete its current grant within a reasonable amount of time.

Evidence: Performance Work Plans for Administrator and Deputy Administrator, Grant Agreements, Grantee Performance and Financial Reports, site visit reports, 7 CFR 4284, 2006 Notice of Solicitation of Applications.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Though the VAPG allocates funds shortly after the announcement of award winners, and obligates funds within 90 days of the award announcement, it takes Cooperative Programs an average of 501 days to finish and close a project. Some grants funded in 2001 are still currently open. To deal with this time disparity USDA has added a long term performance measure to minimize the number of accounts still open after 18 months. Program regulations require initial screening of grant applications to verify applicant eligibility, realistic work plan for use of funds, verification of matching funds. Because of the low grant amounts awarded in recent years, most Grantees are not subject to an A-133 audit. However, these Grantees are still required to have a limited scope audit conducted by a CPA or an RD employee, depending on the Grantee and the amount of funds awarded. Any findings from any audit are immediately forwarded to the National Office or referred to the Office of the General Council for further action. 7 CFR 1951 and the associated RD Instruction govern unauthorized assistance procedures.

Evidence: 7 CFR Parts 1951 and 4284 govern program administration, including eligibility and servicing of grants. GLS obligation and disbursement reports identify when funds are obligated and disbursed.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: One annual efficiency measure (cost per application processed) and one long-term efficiency measure (three year average of the percentage of grants completed within 18 months of obligation) are in place. These measures assist Cooperative Programs with determining efficiency in processing applications received and assuring grants are completed in a timely fashion. To ensure effectiveness and acheive efficiencies CP has begun work with internal IT staff to develop a secure, online, application processing system. Work on the first component of the system - a secure website that can be accessed by USDA and non-USDA approved personnel - was completed in April 2005. The Agency is currently setting up a business case with IT personnel to itemize the tasks needed for completion and set priorities. This work internalizes a previously outsourced activity with a potential savings of $240,000 per year. Beginning in 2003, the Agency also allowed applicants to submit applications electronically. With the implementation of Grants.gov, the Agency has required electronic applications to be submitted through Grants.gov. It is currently estimated that processing an electronic application saves almost $3 over processing a paper application. Unfortunately, due to the nature of the applicant pool for the program and the complexity of submitting applications through Grants.gov, the Agency does not expect to require all applications to be submitted electronically. Other cost savings are currently being considered. A draft of a revision to the existing program regulation is being considered at this time. The revisions, would, in part, reduce the burden to the applicant in terms of requiring a less complex document, and would also reduce the burden to the Agency because of reduced processing time. The final cost savings will not be calculated until the revision has been cleared, but it is currently estimated that the reduction in burden could reduce processing time by up to 50%.

Evidence: Performance measures are entered in the Department's performance measure tracking system, internal Agency cost records, GLS reports, Grants.gov, previous contracts for application processing.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The VAPG program works in conjunction with other RD programs to stimulate economic development in rural areas. While these other programs are not specifically designated as value-added agriculture and are limited to rural areas, there is some overlap. Many RD programs, including VAPG, share a performance measure of creating jobs. RD administers almost all of its programs through its State and local offices. Resources are allocated in these offices so that potential applicants for all RD programs can obtain information about programs and apply for assistance at the same office, often through the same contact person. For example, most of the VAPG State Office contacts also work on other RD programs such as the Business and Industry Guaranteed Loan program and the Renewable Energy and Energy Efficiency program (REEEP). Thus, if a local agricultural producer comes to the State Office, the same contact person can go over several programs with the applicant to determine which program best meets the applicant's needs. The same type of coordination and resource allocation occurs at the National Office level. Even though RD program staff often specialize in certain programs, they are all aware of the other programs available, so that referrals can be made among the programs and programs can be run as consistently as possible. An example of National Office coordination occurred during the drafting of the 2006 Notice of Solicitation of Applications for the VAPG program. Because one type of eligible project for the program involves renewable energy and RD has a separate program for renewable energy - REEEP, the VAPG NOSA borrowed the definitions of several terms from REEEP so that the administration of the two programs would be easier for RD personnel involved with both programs and so that program delivery would be more consistent.

Evidence: USDA Strategic Plan, Rural Development Strategic Plan, Rural Development Performance Budget Initiative, RBS website with program descriptions (http://www.rurdev.usda.gov/rbs/index.html).

Does the program use strong financial management practices?

Explanation: VAPG regulations and RD Program Instructions provide guidelines for the financial management practices. Both National Office and State Office personnel must use the RD General Ledger System (Grants and Loans Financial Management System) and the FFIS to ensure that payments are made properly for the intended purpose to minimize erroneous payments. Both of these financial management systems meet statutory requirements.

Evidence: The GLS system is audited and no material weaknesses have been found. The program has procedures in place to ensure that payments are made properly for the intended purpose to minimize erroneous payments. Program Regulations:§ 4284.908 Use of grant and matching funds.(3) Hire a Certified Public Accountant or other qualified individual to design an accounting system for the proposed venture'.§ 4284.910 Application processing. (6) Verification of Matching Funds.§ 4284.913 Evaluation criteria and weights.(5) Work plan/Budget. (6) Amount requested.§ 4284.914 Grant closing. (a) Letter of Conditions.(b) Applicant's intent to meet conditions.(c) Grant agreement. Additionally, OIG is completing an in-depth audits in two areas -- (1) Are the entities that have received grants eligible to receive them and (2) are we sufficiently monitoring grantees in terms of use of grant funds and performance.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: There are currently four mechanisms that are used by the VAPG program to identify and address management deficiencies. One of these mechanisms is completely internal to the program staff and the other mechanisms involve non-program staff. Internally, program staff and management meet regularly to outline both short-term and long-term strategic goals and to set time frames for achieving those goals. Six performance measures have been approved by Cooperative Programs and reporting mechanisms are identified for each measure. These performance measures will help program staff identify progress towards achieving long-term goals. These staff also are responsible for revising regulations and taking stakeholder feedback regarding the program. Outside of the program staff, the Office of the Inspector General is currently conducting a three-phase audit of the VAPG program. The audit process is formalized so that any deficiencies identified during the audit require a response from the Agency explaining how it will correct the deficiencies. The Financial Management Division (FMD) of RD provides oversight to ensure that the Agency follows through with its corrections. Also, the program is currently scheduled for a Management Control Review (MCR) in FY 2008. MCRs strive to determine whether necessary controls are in place and producing the intended results, comply with applicable laws and regulations, and provide solutions to reduce or eliminate any deficiencies. The recommendations from the MCR will be incorporated into the program when the MCR is completed. MCRs are conducted once every 3-5 years. Again, the FMD provides oversight. Finally, the VAPG program is scheduled to be included in the Program Assessment Review (PAR) process beginning in FY 2007. This process is conducted by the newly created Oversight and Coordination Services (OCS) division of Business and Cooperative Programs. OCS has developed a manual to provide instructions on conducting the PARs. While MCRs focus on the National Office management controls, PARs focus on State Office management controls. Each year, approximately 10 State Offices go through a PAR, so each office is assessed once every 5 years. OCS will provide oversight to ensure the PARs are conducted by non-program personnel and that deficiencies are identified and addressed.

Evidence: USDA MITS, (the Department's performance measure tracking system), National Office communication to State Offices regarding reporting of performance measure data, GLS reporting system, performance reports from State Office, OIG Audit Report No. 34601-3-KC located at http://www.usda.gov/oig/webdocs/34601-3-KC.pdf, RD Instruction 2006-M (http://teamrd.usda.gov/rd/cfo/fmd_home.htm). The long-term strategic goals include: publishing processing and servicing Administrative Notices, so that the field offices will have improved guidance on administering the program; publishing a revised program regulation that reduces the burden for both applicants and the Agency; updating and enhancing the program website to include more information for applicants on applicable regulations and application tips; being prepared to accept applications and announce awards earlier in the fiscal year; to conduct training for field office staff on monitoring grants.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The program requires a national competition with clear eligibility and application requirements that are published in the program regulation and an annual solicitation. The requirements are also presented in a more informal guide format combined with an electronic application template and checklist to assist applicants that are not experienced with applying for Federal grants. Before an application is considered for funding, it must demonstrate that the applicant and project are eligible and it must be complete. 100 percent of funding is awarded according to a competitive process. Independent reviewers conduct independent merit reviews and score applications. Scores are then adjusted by using an accepted statistical procedure that adjusts for scoring bias. Applications are ranked based on the average, standardized score, and funds are awarded in rank order until available funds are exhausted. Program outreach is conducted in a variety of ways. Applications are solicited using an announcement in the Federal Register. Program information is also disseminated to Grants.gov and the Catalog of Federal Domestic Assistance. A program website is maintained that includes instructions for applying, tools for assembling a complete application, contact information, and lists of previous grantees. RD State Office staff also promotes the program by conducting presentations to potential applicants, and facilitating local news coverage of the program. While Grantees are eligible to re-apply for additional grants, their eligibility is restricted to two grants per project (one planning and one working capital), and applicants applying for a grant for a new project are penalized if they have already received one or two grants for a different project. No formal report is available on the number of grantees that have received grants in multiple year, but it is estimated to be at less than 10%. Grants are awarded for a one year time period. A no-cost extension may be approved if there are good reasons for a delay in progress, but grants are never renewed or provided additional funding beyond the original grant award.

Evidence: The program regulation, 7 CFR 4284, can be found at http://www.gpoaccess.gov/cfr/index.html. It and the 2006 NOSA outline the competitive process that is used to evaluate applications and award grants. The NOSA can be viewed at: http://a257.g.akamaitech.net/7/257/2422/01jan20051800/edocket.access.gpo.gov/2005/pdf/E5-7596.pdf. The program website is located at: http://www.rurdev.usda.gov/rbs/coops/vadg.htm.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The program has oversight capacity in several ways. All grantees must submit performance and financial reports on a semi-annual basis. Each Grantee is visited at least once during the course of the grant period to confirm that the reports submitted are accurate and to review records. Grantees are subject to some level of audit requirements, depending on the amount of the grant, the Grantee's past performance in other Federal programs, and compliance with 7 CFR 3052. Disbursements are made on a reimbursement basis only, with documentation required with each request for reimbursement showing what activities were completed.

Evidence: 7 CFR 3015, 3019, 3052, and 4284 provide information on allowable uses of funds, oversight practices, including site visits and record retention, as well as audit requirements. The Grant Agreement reminds Grantees of their reporting obligations, the Agency's right to conduct site visits, and their audit requirements. OMB Circular A-122 and the Federal Acquisition Regulation, Section 31.2 are the governing cost principles for non-profit and for-profit grantees, respectively. Grantee performance and financial reports demonstrate the Grantees' adherence to the approved budget and work plan. Grantee requests for reimbursement demonstrate the documentation required to prove that only approved expenditures are paid with grant and matching funds.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program collects data on a continuous basis for some performance measures and at least annually for all other measures. The program expects to make non-confidential data available to the public through the USDA RBS program website once the baseline for the performance measures has been set in June 2006. Each measure will be reported individually by year on the website. This information will include, their target, baseline, and actual data. The website currently highlights individual successful projects and lists all recipients by year. The publication, Rural Cooperatives, is a bi-monthly magazine featuring a success story each issue about a VAPG recipient. The magazine is available on the web for no charge and by subscription.

Evidence: Data for the jobs and grants completed measures are currently entered in GLS, with other data collected via e-mail at this time. A Request for Automation has been submitted to the IT staff requesting updates to GLS to allow data to be entered by the field for all other measures except the cost per application measure, which is captured and calculated by the National Office. The program website can be found at: http://www.rurdev.usda.gov/rbs/coops/vadg.htm. It currently includes a Success Story section which highlights individual projects. This section will be expanded in the future to include aggregated performance measure information for the six approved measures for the program. The Rural Cooperatives issues containing the Value-Added Corner articles can be viewed at: http://www.rurdev.usda.gov/rbs/pub/openmag.htm.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Congress established the enhancement of producer revenues as one goal for the VAPG program. The agency has been working hard, creating ambitious baselines and targets for this long term performance goal. FY 2005 Grantees are scheduled to complete their grants in the fall/winter of FY 2007; achieving annual performance goals is unfeasible. The second performance measure was established to track the long-term viability of the grantees following the award. The preliminary report indicates that USDA has not met their 2002 target of 35%, however the 2001 and 2002 grants have not been completed for at least three years for the given fiscal year, the actuals are subject to change and will be updated in the next year.

Evidence: See Measures on PARTweb. National Office communication to the Rural Development State Offices, GLS reports, program performance data collection, PART measures, USDA Budget and Performance Report, State Office reports, see website for an example.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The agency has been working hard, creating ambitious baselines and targets for their annual performance measures. FY 2005 Grantees are scheduled to complete their grants in the fall/winter of FY 2007. Due to this time constraint USDA has not been able to get data on three of their four annual performance goals. Once the data becomes available, in late winter/early spring, the program expects to achieve their annual targets. Since the efficiency measure, cost per application processed did not reach its target goal of $1,000 and the other annual measures do not yet have data, the answer to this question is no.

Evidence: See Measures in PARTweb. National Office communication to the Rural Development State Offices, GLS reports, USDA Quarterly Performance Report, RD Budget and Performance Reports.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: USDA has not reached its annual efficiency measure goal for 2006, of cost per application processed. It has not had enough time to collect data on its long term efficiency measure, the three-year average of the percentage of grants completed within 18 months of obligation date. However the program has taken dramatic steps in improving efficiencies in the last few years, internalizing (as of 2005) previously outsourced IT procedures, which resulted in an overall savings of approximately $240,000 per year. The program has internal efficiency measures to gauge program efficiencies and cost effectiveness. Full cost reports are calculated for Budget and Performance Reporting.

Evidence: See PARTweb performance measures. IT agreement, DynAccSys contract, internal Agency cost data. Full Cost assessment in Budget and Reporting Reports.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There is no program that provides the same significant level of funding to a producer for efforts toward value-added agriculture. However, the VAPG program works in conjunction with other USDA programs to stimulate economic development in rural areas. There is some overlap among RD programs. The ultimate goal of nearly all USDA RD programs is to enhance the economic well-being of rural areas. Other USDA RD programs may be able to fund similar or related projects. For instance, the Renewable Energy and Energy Efficiency Program (REEP) focuses specifically on energy projects and provides both loans and grants. Energy projects may also be funded through the VAPG program, although they may not be the same projects that are eligible for the REEP. RD State and local offices coordinate to work with potential applicants to ensure that the applicant applies for the correct type of funds at the correct stage in its development. Currently, the VAPG program is looking at targeting an increasing portion of program funds to emerging markets. This targeting will improve the program's performance and in the long term, increase its score.

Evidence: USDA Strategic Plan, Rural Development Strategic Plan, Rural Development Performance Budget Initiative, RBS website with program descriptions (http://www.rurdev.usda.gov/rbs/index.html).

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: There is three phase audit being conducted by USDA OIG. In phase one, the VAPG program was subject to OIG review of National Office procedures in 2003. OIG examined all data and all procedures conducted by the National Office. The audit report was issued in 2004, and indicated that the internal controls for the program were lacking at that time. However, the Agency has made vast strides since the issuance of the report, including publishing a program regulation at 7 CFR 4284. Currently, OIG review is looking at State Office monitoring of existing grants. In this phase, they conducted a stratified, random sample to select State Offices to review. In phase three, OIG will review applicant eligibility to examine whether the grants awarded were awarded to eligible entities. In this phase, it is expected that the same type of stratified, random sampling technique used in phase two will be used. Internal to RD, the Financial Management Division conducts both Management Control Reviews (MCR) and State Internal Reviews (SIR). MCRs strive to determine whether necessary controls are in place and producing the intended results, comply with applicable laws and regulations, and provide solutions to reduce or eliminate any deficiencies. The VAPG program is scheduled for an MCR in FY 2008. SIRs are conducted in the same manner but focus on program delivery at the state level to assure programs are operated correctly by RD field staff. The VAPG program is scheduled to be included in the SIR process in FY 2007. While RD staff is used to conduct the MCRs and SIRs, they are not staff working on the programs examined. Staff from both the National Office level as well as the State Office level is selected. The Agency has also entered into a Grant Agreement with the University of Missouri to assess the impact of the VAPG program using information from a survey given to Grantees from 2001 through 2003. It is important to note that this grant was awarded through a nationally competitive process. At this time, surveys have been sent to all grant recipients from those years. The deliverable will include recommendations for performance monitoring, challenges faced by Grantees in completing their projects, and an assessment of the impact of the program in demand for agricultural products, market prices, farm income, and Federal outlays on commodity programs.

Evidence: The OIG audit report for phase one can be found at: http://www.usda.gov/oig/webdocs/34601-3-KC.pdf. 7 CFR 4284 contains program eligibility information as well as processing and servicing requirements. It can be found at: http://www.gpoaccess.gov/cfr/index.html. RD Instruction 2006-M explains the MCR and SIR process and can be found at: http://teamrd.usda.gov/rd/cfo/fmd_home.htm. The Grant Agreement with the University of Missouri describes the deliverables and due dates for the work.