Detailed Information on the
Research/Extension Grants: Economic Opportunities for Producers Assessment

Improving the focus of grant recipient reporting to place more emphasis on outcomes.

Increasing planning and coordination with the Agricutural Research Service regarding the collection of stakeholder input.

Continue to improve its long term measures for these programs.

Emphasized funding through competitive grants, by proposing to increase the National Research Initiative (NRI) from $181 million in 2006 to $248 million in 2007, and increasing competitive grants through the Hatch and McIntire Stennis programs.

Modify the long term measures to show the actual use of the results of research, rather than just the number of accomplishments developed.

Improve efficiencies in the grant review process.

Continuing to improve efficiencies in the grant process, throught the utilization of "Grants.gov," the web-based Peer Review System, and virtual panels where appropriate.

Measure: Cumulative number of biochemical or thermochemical technologies which are developed and used commercially for the conversion of biomass to fuels.

Explanation:Commercially adaptable processes to convert cellulose to sugars, fermentation of cellulosic sugars to Ethanol, chemical transesterification of oils from oilseed crops, and the thermal pyrolysis and gasification of biomass will have been increased by 2009. These will increase the biofuel conversion and utilization for the US consumers. This measure captures the goal to increase the use of biomass-based transportation fuels 8 folds over the next 5 years. Data from the Renewable Fuel Association shows the use of fuel ethanol increasing from 174M gallons in 1980 to 2.8B gallons in 2003. Current data show that 3.4B gallons were produced in 2004 and the Association reported that there is capacity to produce 4.4B gallons in 2005. The National Biodiesel Board estimates usage has increased from 500,000 gallons in 1999 to 75M gallons in 2005.

Measure: Cumulative number of new crops that have been developed and used commercially.

Explanation:New crops provide agricultural diversity, new sources of revenue and can be grown sustainably with reduced inputs.

Measure: Portfolio Review Score

Explanation:Goal performance is externally assessed on a 5 year basis to determine progress toward solving targeted national problems related to each Strategic Objective. The score reflects expert assessment on 3 PMA dimensions of relevance, quality and performance, along a continuum from exceeds expectations, to meets expectations, to needs improvement. The reported score is a mean of quantitative scores for the Goal's objectives."

Measure: Cumulative Dollars Saved for Grant Review.

Explanation:Dollars saved reflects the average salary saved by calculating the number of calendar days saved annually between receipt of proposal and date funding awarded for competitively reviewed proposals, then multiplied by the average daily salary for CSREES employees.

Measure: Proposal Review Time in days

Explanation:Average number of calendar days between receipt of proposal and date funding awarded for competitively reviewed proposals. Average is calculated by summing the total number of days for all major competitively reviewed proposals and dividing by their total number.

Measure: Internal Portfolio Review Score (New measure, added February 2007)

Explanation:Goal performance is internally assessed annually by NPLs to determine progress toward solving targeted national problems related to each Strategic Objective. The score reflects expert assessment on 3 PMA dimensions of relevance, quality and performance, along a continuum from exceeds expectations, to meets expectations, to needs improvement. The reported score is a mean of quantitative scores for the Goal's objectives.

Is the program purpose clear?

Explanation: The purpose is to enhance economic opportunities and the performance of agricultural producers by increasing productivity, market and trade opportunities and management. The agency program is fully aligned with and meets the mission for USDA Strategic Goal 1 (Enhance Economic Opportunities for Agricultural Producers), which is to advance the creation and distribution of science-based knowledge specifically to enhance economic opportunities open to agricultural producers. The Cooperative State Research, Education and Extension Service's Goal 1 objectives are met through coordinated work under 6 portfolios: Expand International Marketing Opportunities; Support International Economic Development and Trade Capacity Building; Expand Alternative Markets for Agricultural Products and Activities; Risk Management and Financial Tools to Farmers and Ranchers; Promote the Efficiency of Agricultural Plant Production Systems; Promote the Efficiency of Agricultural Animal Production Systems. Education, research and extension work with multiple funding authorities in all 6 portfolios is appropriately integrated and coordinated, and is designed to enhance economic opportunities and performance of agricultural producers by increasing productivity, market and trade opportunities, and management.

Evidence: CSREES education, research and extension work to enhance economic opportunities for agricultural producers is authorized under Food Security and Rural Investment Act of 2002; Competitive, Special, & Facilities Research Grant Act; Agricultural Research, Extension, & Education Reform Act of 1998; Federal Agriculture Improvement & Reform Act of 1996; National Agricultural Research, Extension, & Teaching Policy Act of 1977; National Agricultural Research, Extension, & Teaching Policy Act Amendments of 1985; National Aquaculture Act of 1980; Food, Agriculture, Conservation, & Trade Act of 1990, Title XVI; Food, Agriculture, Conservation, & Trade Act of 1990, Title XXV; National Nutrition Monitoring & Related Research Act of 1990; Rural Development Act of 1972; Research Facilities Act; Conference Report for Agricultural Risk Protection Act of 2000. Goal 1 is clearly defined by the UDSA, REE Mission Area, and CSREES Strategic Plans, the CSREES Mission, and annual and revised USDA Performance Plans.

Does the program address a specific and existing problem, interest or need?

Explanation: Maintaining agricultural sector economic viability has been a priority of Congress and every Administration since the Great Depression. CSREES addresses existing national needs to enhance economic opportunities for agricultural producers through sponsorship of education, research, and extension. Preserving, maintaining and expanding plant/animal genetic diversity is crucial to adapt quickly to changing conditions. Identifying and managing risks inherent in agriculture helps strengthen competitiveness and maintain sustainable income and profitability. Improving and developing new food and non-food bio-based products, including fuels, is critical to respond to domestic and global demands. Analysis of domestic and international policy and developing sound management and marketing responses helps guide development and ensure individual producer use of competitive, economically sound, environmentally friendly approaches to produce products national needes for food, feed and fiber.

Evidence: CSREES Goal 1 directly addresses specific national needs to enhance economic opportunities for agricultural producers, as per the USDA and REE Strategic Plans. Goal 1 work focuses on Agricultural Markets and Trade, International Economic Development, Agricultural and Food Processing/Biobased Products (including fuels), Structure of the Agricultural Sector and Farm Management (including risk management), Plant Production, and Animal Production collectively to enhance economic opportunities for agricultural producers.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: CSREES' unique mission to integrate education, research, and extension extramural programs is coordinated with federal, state, local and private sector to avoid excessive overlap, redundancy or unnecessary duplication. Integrated sponsorship of knowledge generation, education, and technology transfer is not duplicated by others and is carried out with university partners. Diversity of programs and funding allow applied research and technology transfer for immediate response to emerging needs, and longer term work to develop and transfer information to producers to deal with the underlying causes of problems by adjusting production and marketing methods and using new technology. The scope of work is integrated and complimentary with USDA agencies (e.g., ARS, ERS, NASS, NRCS, APHIS, FSA, RMA, GIPSA) other departments and institutions (e.g., NSF, DOE, EPA, NASA), states, and private sector to provide inclusive, but non-duplicative, coverage. Coordination adds value to agency work and the related work of other providers in the public and private sector.

Evidence: Goal 1 objectives enhance economic opportunities for agricultural producers through coordination and consultation and input from other agencies and stakeholders, and in cooperation with partners. National Program Leaders coordinate and collaborate with partners, other providers and constituents to ensure work is not redundant or duplicative but is complementary and supplementary to ensure comprehensive, validated knowledge is developed and adopted by relevant constituents. This is done through meetings with counterparts (e.g., ARS scientists, ERS economists), seeking input for Requests for Applications that complement and enhance the value of work of other providers, and including them on peer reviews panels. For example, joint CSREES/ARS work on soybean rust includes fundamental research to understand nature of the disease, develop short term responses to infestation, prepare agents to contain or control outbreaks, develop resistant varieties, develop new production methods, and quantify the value of damaged crops.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: This answer received a "no" despite the following write-up, which shows that CSREES has a strong internal merit- review process. It received a "no" because a portion (less than 10 percent) of programs are earmarked through Congress appropriations to a specific location and purpose. CSREES has designed all portfolios to effectively address needs of agricultural producers. They integrate areas of science, multiple funding sources, and education, research and and extension efforts in a comprehensive, non-duplicative package. This approach optimizes the strengths, targets and benefits of a complex mix of authorizations, 57 funding lines, and traditional programs around common areas of need and science. Each component, regardless of funding activity, is held to high standards of relevance, quality and productivity specificed in the President's Managemnt Agenda. All components, including formula, special grants, earmarks and integrated programs (that is programs that have both research and extension components) submit proposals or work plans that are independently peer- or merit-reviewed by internal National Program Leaders (NPL's) and external panels or appropriate experts prior to the release of funds. Indpendent of funding mechanisms, funding id not awarded to inferior proposals, and funding is deferred pending correction of deficiencies.

Evidence: NPLs use scientific expertise, independent review panel and stakeholder input and other info. to prepare guidance for education, research and extension activities to address national needs and problems. This guidance reflects changes in needs, the state of science and education, administration and congress priorities to ensure comprehensive, effective, efficient investment of public resources to enhance economic opportunity for producers. Peer/merit review ensures relevance, quality and productivity of all funded activities, including formula and special grants. Integrated education, research and extension work consists of explicit problem areas that clearly define areas of science and education, e.g., Animal Production focuses on Reproductive Performance, Nutrient Utilization, Genetic Improvement, Animal Genomics, Physiological Processes, Environmental Stress in Animals, Production Management Systems, and Improved Animal Products. Topics of animal protection, as distinct from production are covered in Animal Protection in Goal 3.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: CSREES portfolios are targeted to concentrate research and education on enhancing productive capacity, management and marketing issues specific to agricultural producers. The combination of authorizations and funding streams targets extramural education, research and extension at institutions with broad geographic coverage, and include designated resources for minority and other institutions that target specific populations. This helps ensure a comprehensive, continuous flow of science based knowledge and technologies that are relevant to the diverse needs of multiple constituencies.

Evidence: CSREES designed and packaged Goal 1 programs in collaboration with other agencies, including ARS and ERS to complement work of other agencies and to supplement work not covered in their mission areas to avoid redundancy and duplication of efforts and funding. CSREES' unique mission to support research, and education, and extension is an asset in specifically utilizing and adding value to the research efforts of other federal agencies and departments as well as that of other public and private sector providers.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: CSREES has a balanced strategy to deal with problems and topics to enhance economic opportunities for U.S. producers. Strategic Goal 1 has a limited number of quantifiable long term outcome based performance measures that focus on ensuring the profitability of agriculture through improved production, management and marketing, and providing an adequate, affordable supply of products to meet domestic and global needs for food, fiber, and related bio-based products.

Evidence: The long term goals established for Goal 1 are: (1) Increase use of transportation fuels from biomass to 4% of total transp[ortation fuel use by 2010, (2) the Agricultural Productivity Index (crops and animals) will increase by 123 by 2010. Data for these measures come from ERS and Office of Energy Policy and New Uses.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The long-term impact of CSREES Education, research and extension is increased agricultural productivity. The long term (2010) goal is a rate of growth that exceeds the Post World War II historical average. Between 1948-1994 ag. productivity rose at a 1.88% annual rate. During the 1990s, as average annual growth rates for federal obligations for basic research declined (Federal Funding of Basic Research www.ams.org/government/growthrates.html) annual productivity growth was 1.15%. CSREES and its sibling agencies in the REE Mission Area, as well as Forest Service research agencies, aligned their strategic plans with REE and USDA and coordinated similar long term objectives and problem areas to increase productivity, value added, and returns to public education, research, and extension.

Evidence: Long-term performance is assessed on a 5 year basis to better comprehend the cumulative nature of research and education processes, and to consider the aggregate flow of outcomes and impacts that may be sporadic and have time lags that are impossible to predict ex ante. The CSREES target is an average annual productivity growth rate through 2010 that exceeds long tern and recent historical trends.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: CSREES has specific annual performance and efficiency targets that measure progress toward achieving long term goals. Annual measures include output (e.g., quality, relevance and productivity of education, research and extension work to increase agricultural productivity) and efficiency (e.g., cost and time to review proposals and award funding to grantees).

Evidence: Annual performance of the Goal 1 objectives is assessed to determine progress toward solving targeted national problems and achieving long term goals. Performance review scores for each objective reflect expert assessment of the three PMA dimensions of relevance, quality, and performance, along a continuum from exceeds expectations to meets expectations, to needs improvement. The reported score is a mean of quantitative Portfolio Review Panel scores for Agricultural Markets & Trade, International Economic Development, Agricultural & Food Processing/Biobased Products (including fuels), Structure of the Agricultural Sector and Farm Management (including risk management), Plant Production, and Animal Production. Efficiency measures monitor progress in reducing cost and time per proposal processed and reviewed.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Following the recent completion of the USDA, REE and CSREES Strategic Plans, portfolios of work were established for each Strategic Objective. Annual measures for performance and efficiency have been established. Baselines are established on recent performance, and ambitious targets have been established to routinely and continuously improve performance, reduce costs, and save time.

Evidence: Annual performance scores for each objective are baselined at meets expectations. Annual performance targets call for a cumulative 5 percentage point improvement in exceeding expectations. Efficiency baselines for cost per proposal reviewed ($356) and mean time to review proposals and release funds (161) days were established for 2002. Annual targets call for an average 3% reduction in review and processing cost, and an average 3% reduction in review time.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: CSREES enters into education, research, and/or extension partnerships only when mutual benefit is clearly demonstrated and it is consistent with CSREES mission, Strategic Plan and long term and annual goals. Submitted proposals and plans of work specify the scope of work, milestones, measures, outputs and outcomes, along with performance monitoring procedures. Submission of proposals and acceptance of awarded funds requires contractual commitment to working to accomplish annual and long term goals.

Evidence: CSREES funds over 2000 proposals, cooperative agreements and other transactions yearly. All proposals and plans of work (including formula, special, other mandated funding) must include measures relevant to CSREES long term and annual performance goals and targets. NPL review and monitoring of performance assures continued commitment of partners. Funds are withheld or deferred in the event that the contracted commitment is not maintained. As an example of government partners working toward common goals, over the past 3 years Cooperative Extension Service partnering with Farm Service Agency served a useful role in building a mutually beneficial working relationship between government agricultural agencies in need of production training for recently hired Farm Loan Officer Trainees www.joe.org/joe/2004april/ent.shtml#rb5.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: All Goal 1 portfolios are independently assessed by panels of outside experts every 5 years. Assessments provide useful comments to help NPLs and implementing partners generate meaningful outcomes.Other external reviews and evaluations of CSREES programs are encompassed in materials reviewed by the review panels in making their assessment of the portfolio. For example, multiple evaluation studies in their entirety can be submitted as evidence of effectiveness and accomplishments.

Evidence: The results of independent expert portfolio review assessments for CSREES Goal 1 can be found in the Long Term Measures tab. Also see individual evaluations, e.g., Battelle Memorial Institute, "Ohio Agricultural Research and Development Center: A Generator of Positive Economic Impacts for Ohio". www.oardc.ohio-state.edu/bab.pdf

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Budget and Performance Integration (BPI) process ensures a direct link between long range and annual goals, defined in the 3 Strategic Pans, and the annual Departmental Performance Plan. Designation of explicit problem areas that define the portfolios are exclusive and non-duplicative and make it possible to directly link budget lines with goals, objectives, portfolios, and problem areas in a transparent manner.

Evidence: See Agency budget requests, Explanatory Notes and accompanying information. New classification and reporting systems currently under development or in pilot testing will strengthen the budget and performance linkages and increase transparency.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Significant efforts were undertaken to fully align the new CSREES Strategic Plan with the REE and the USDA Strategic Plans. Comprehensive Problem Area codes have been developed for all Agency work and are being used to align areas of CSREES higher education, research, and extension work under the portfolios that support the strategic objectives. Budget Performance Integration enhances budget preparation and justification based on need, strategic planning, and overall performance.

Evidence: CSREES demonstrates comprehensive linkages of all Goal 1 portfolios between congressional authorizations and appropriations, the Strategic Plans, the departmental and agency mission, the departmental Performance Plan, and specific areas of science and education.

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: The Goal 1 program is continuously coordinated through CSREES NPL consultation with complementary programs in USDA, other departments, universities, state agencys, industry, producers and other stakeholders. Program relevance to the departmental and REE missions is regularly evaluated and coordinated with other providers and partners. NPLs also provide information guidance and counsel to other agencies, scentists, providers and advisory groups to assess and compare the benefits of the work of similiar groups.

Evidence: Constituents, stakeholders and partners are routinely involved in program planning, development and review. NPLs consult with academic and industry scientists and educators beyond the Department, with advisory groups (e.g., National Agricultural Research, Extension, Education, and Economics Advisory Board; Council for Agricultural Research, Extension, and Teaching; Council for Agricultural Science and Technology) and industry groups (e.g., national producers' associations; farm organizations) to assure the relevancy of education, research and extension and to ensure complementarity with the programs of others. Considerable coordination takes place with USDA, interagency work groups, and other departments (e.g., USDA/NASA Interagency Working Group; Interagency Working Group on Plant Genomes; Buy Bio Interagency Work Group) to add value and extend the benefits of the efforts of CSREES' and other programs with similar goals.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The agency develops budget and programmatic priorities based on existing and emerging needs, recent success in research, the state of science, educational capacity, and direct input form partners, constituents and other providers. The comprehensive BPI process makes the linkages and potential gaps and overlaps that become apparent as the budget request is being prepared, allowing for refinement of the budget request before it is submitted.

Evidence: NPLs and their colleagues from other agencies and institutions collaborate on the development of RFAs, including priorities, the direction of science and education, and the integration of portfolio work within the agency. This information is an integral component of the BPI process in collaboration with OBPA and OMB.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Recipients of competitive and line item grants are required to submit annual progress reports and final reports through the Current Research Information System (CRIS) for research, higher education and integrated activities or directly to the NPL as per Terms and Conditions of grant awards. NPLs use this information to monitor progress, make recommendations, measure impact and make changes in priorities & focus, and program administration. CSREES collects information on the progress of research formula awards through CRIS and collects information on the total portfolio of agricultural research and extension activities through the plan-of-work reporting process and other proprietary databases, with NPL oversight. Extension work is reported through Plans of Work and annual reports. CSREES is developing and implementing a reporting system or add-on feature to CRIS for extension work. This will better integrate performance data across the areas of education, research and extension.

Evidence: Terms and Conditions for competitive and line item awards specify annual and final report requirements. Reports determine progress of each grant and if additional grant funds should be provided. NPLs use information to compile reports, measure program impacts & respond to inquiries from management, public, and Congress. See http://cris.csrees.usda.gov/. Research projects supported with formula funds are reported in CRIS as per Administrative Manuals for Hatch, Evans-Allen, McIntire-Stennis & Animal Health Research Programs. Some extension formula programs have proprietary database reporting systems. 1862 and1890 Land-Grant Institutions are required to submit 5-Year Plans of Work and Annual Reports of Accomplishments and Results (See Guidelines for State Plans of Work & Annual Report Guidelines). Reports measure impacts of institutions in addressing critical issues identified through stakeholder input.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: CSREES managers are accountable at all levels of the agency. The Office of Extramural Programs (OEP) is fiscally accountable, and program managers are subject to Financial Assistance Regulations. Partners submit Cash Transactions and and Financial Status Reports and undergo audit, per OMB Circular A-133. Formula grantees submit a Cash Transactions Report and annual Financial Status and documentation to sustain the report. Agency NPL's are accountable for schedule and performance results. NPL's monitor progress by evaluating and measuring program impacts. CSREES performance standards relate to strategic goals and objectives. Project periods (three to five years) ensure time to carry out work on schedule and as approved from peer- or merit- reviewed proposals and plans of work. NPL's consider budget adequacy; partners obtain prior approval for budget changes. NPL's review and approve plans of work from the land grant institutions, and arrange for and coordinate review of any programs not formally covered by plan of work reporting.

Evidence: Budget Office develops Control Tables with NPL assignments for the fiscal year. Office of Extramural Programs develops guidelines and training to assist NPLs in solicitation, review, and award of grants. This is one effort of PACE Committee, a CSREES workgroup of NPLs and administrative staff working to streamline and simplify the grants administration process and to solve problems of mutual interest. Grant recipients are provided an annual allocation letter that provides amounts of formula grant awards, and if applicable, matching requirements. Institutions with a match requirement submit a Certification of Offset form before formula funds are released. Guidelines for State Plans of Work, Annual Report Guidance, and Administrative Guidance for Integrated Research and Extension Activities provide formula grantees with compliance and reporting instructions for the administration of most CSREES formula funds.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds are, in most cases, required to be obligated in the year appropriated. Audits, reviews and mandatory reporting, along with active post-award management, ensure that funds are spent for the intended and agreed upon purposes.

Evidence: CSREES prepares a schedule of obligations,monitors unobligated balances, compares obligations with scheduled estimates, and makes appropriate asjustments. NPLs can defer the dispersal of obligated funds if deficiencies or deviations are detected. Awardees are required to take corrective action in order to secure the release of funds.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: CSREES has an internal grants management system, CREEMS, which assists management and staff by providing budget, expenditures, and statistical data related to the administration of grants (e.g., time to process a proposal from receipt to award; time to process a post-award action). The system serves as a management tool for tracking action items and administrative compliance requirements. The system is CSREES' innovation in integrating all grant processes (programmatic, administrative and fiscal) into one database. Annual efficiency measures compare proposal review time and costs.

Evidence: See CREEMS documentation www.csrees.usda.gov/business/other_links/egov/csrees/creems.html. See Annual Efficiency Measures tab. The agency has a comprehensive IT Strategic Plan to increase the use of technology to increase cost and time efficiency, www.csrees.usda.gov/about/offices/pdfs/strategic_plan_03-08_Final.pdf

Does the program collaborate and coordinate effectively with related programs?

Explanation: CSREES programs and portfolios are highly integrated and coordinated both within the agency and externally. There is a high degree of coordination between related areas of science, education and extension. Coordination through CSREES NPLs with complementary programs in USDA, other federal departments, universities, state agencies, industry, and with producers and other stakeholders leads to adjustments in program management and resource allocation. NPLs and their counterparts from other federal agencies and institutions collaborate on the development of RFAs and guidance for Plans of Work, including priorities, the direction of science and education, and the integration of multidisciplinary work within the agency and with the work of other providers. Program relevance to the departmental and REE missions and priorities is regularly evaluated and coordinated with other providers and partners.

Evidence: CSREES constituents, stakeholders and partners, along with others doing related work, are routinely involved in program planning, development and review. For example, NPLs collaborate with scientists and educators within and beyond the Department, with advisory groups and industry to maintain the relevancy of education, research and extension and to ensure complementarity with the programs of others. One result is jointly funded programs with NASA, NSF, DOE and others. Coordination takes place within the USDA, with interagency work groups, and with other departments to add value and extend the benefits of the efforts of CSREES programs and of other programs.

Does the program use strong financial management practices?

Explanation: Grantees must comply with Federal financial assistance regulations & specific USDA & CSREES implementing regulations. Audits, reviews & mandatory reporting, with active post-award management, ensure that funds are spent for agreed upon purposes. Grant recipients are required to report expenditures each quarter 45 days after close of the quarter via PSC-272 Federal Cash Transactions Report, via DHHS-PMS. Draw down requests are denied for failure to submit this report. If an institution fails to submit technical and financial reports by June 15th, the 4th quarter formula funds and subsequent formula allocations are withheld until requirements have been satisfied.

Evidence: Financial management is conducted as per OMB Circulars for Federal financial assistance, USDA regulations and the Administrative Manual, CRIS Instructions, C-REEMS instructions, and Agency records indicating the receipt and approval of financial and technical reports. CSREES implemented the Federal Financial Information System in FY 2002. Prompt action is taken to corrrect material weaknesses, if identified by auditors.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Agency has a number of improvement processes underway to upgrade its management efforts. CSREES is in the process of developing guidelines for reviewing officials to use as they review, make recommendations for changes, and approve the project proposals for all formula-funded research. CSREES also has taken action to improve general administrative, managerial, and fiscal functions; post-award management of its entire grant system, including formula, as well as line-item, and competitive grant mechanisms; and planning and accountability efforts. New annual efficiency measures have been developed, with historical baselines and 2004 and 2005 targets.

Evidence: Transition to e-Gov will result in paperless proposal and plan submission, review, approval and funding authorization to reduce processing time. Guidelines for review and approval of formula funding are in review. 5 CSREES programs transition to electronic submission this year. A Program Administration Coordination Effort (PACE) addresses management, administrative, and fiscal issues to increase grant award effectiveness. PACE is completing Service Standards that set temporal guidelines for planning, awards management and time-critical administrative and management activities. A group is establishing guidelines for post-award management. Planning & Accountability provides guidance, analysis, policy recommendations for strategic and short-term planning, budget justification, performance measurement and evaluation, contributing to review and improvement of administrative functions and assisting NPLs assess accomplishments and determine changes to improve performance and efficiency.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Although a portion of grants are appropriated as earmarks, the vast majority of grants are awarded on a competitive-based process. An extensive, rigorous review process for competitive programs includes a panel manager, qualified reviewers, strict guidelines and safeguards to avoid conflict of interest. Formula funded, special or administrative grants are subject to qualified expert assessment of merit prior to award of funding. In all cases, release of funds can be and is deferred pending submission and approval of an acceptable and merit reviewed proposal or plan of work against which performance is assessed. CSREES works with new and first-time recipients to ensure they understand Federal Financial Assistance regulations and policies and works to ensure the proposed project is sound, relevant and meritorious. NPLs monitor awards to ensure progress as intended by legislation and the approved proposal. To ensure success and fulfill requirements, CSREES can place additional controls on new grant recipients.

Evidence: In accordance with to statutes (c.f., Agricultural Research, Extension, & Education Reform Act of 1998) or other policies, proposals and plans of work are subject to either peer review or merit assessment. The agency maintains an extensive data base of highly qualified grant proposal reviewers from government, academia, and industry, and provides strict guidelines, training, and NPL nd administrative oversight to the proposal review and award recommendation process. All reviews are held to the same strict standards of relevance, quality and performance as per the PMA.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The agency has a reporting system (CRIS) in place. Oversight procedures are sufficient to understand and monitor grantee activities. All grantees are required to submit proposals or plans of work fully describing the education, research and extension work for which all types of funding (including mandated formula and special grants) is requested. Grantees are required to submit Annual Reports of Accomplishments describing their progress in achieving their stated goals for each funded activity. Reports are entered in the CRIS data base cris.csrees.usda.gov/ and are available to the public via the Web. NPLs and Awards Management Branch provide review and oversight of all funded activities, including mandated projects. NPLs provide program leadership and oversight, including site visits; agency administrative and support staff assist in program management and support the review process for each program. Expert peer or merit review panels are used to provide assessment of the efforts of grantees.

Evidence: Oversight practices and techniques provide CSREES with knowledge of grantee activities down to the individually funded project level. The Administrative Manual contains explicit program monitoring and reporting requirements. Grantee institutions submit annual Progress Reports; a report is required for each funded project included in the institution's approved work plan for the Federal Fiscal Year. A Termination Report must be submitted for each completed or terminated project. Reports must be submitted in conjunction with progress reports on active projects and must include a summary of outputs and accomplishments over the entire life of the funded project. CSREES is implementing an enhanced reporting system to supplement CRIS to include extension work www.csrees.usda.gov/business/other_links. The agency conducts site visits to a substantial number of grantees on a regular basis, and conducts annual audits of grantee performance. NPLs maintain close working relationships with grantees and their institutions.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: CSREES annually collects, compiles and disseminates grantee performance data down to the funded project level and makes it publicly available in a transparent manner via the CRIS Web Site at cris.csrees.usda.gov/. Grantees are contractually required to submit an Annual Report of Accomplishments, and Termination Reports to describe their performance in achieving their stated goals and to describe their progress on funded projects are entered into CRIS.

Evidence: The widely used CRIS is a comprehensive repository of performance data for funded projects from CSREES and work of related agencies with similias activities. These are identified by problem area and include proposal summaries, annual performance reports, outcomes and impacts, and termination reports. This information is searchable, available to the public, and has the capacity to produce various types of reports according to customer needs. See www.csrees.usda.gov/business/other_links/egov/csrees/cris.htmlAnother CSREES innovation, the Competitive Research, Education, and Extension management System (C-REEMS), makes comprehensive information on proposals, their review and award management available. See www.csrees.usda.gov/business/other_links/egov/csrees/creems.html

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: All partners are required to submit Annual Plans of Work or specific proposals for funding of all types, including formula and special grants. All plans and proposals are subject to merit or peer review in the institution prior to submission to CSREES. All plans and proposals are subject to a competitive-based reviewe by at least 1 NPL and usually by 2 others prior to recommending approval and award of funding. High standards of scientific or educational relevance, quality and productivity apply. Awards Management Branch has procedures similar to those used for competitively awarded grants and funding to maintain the quality of other than competitive grants programs. All funded activities are portions of specific portfolios and are subject to annual and periodic expert portfolio review for relevance, quality and performance. Portfolios of funded projects including formula and special grants are reviewed by independent panels of outside experts for relevance, qualitt and performance.

Evidence: Financial management is conducted as per OMB Circulars for Federal financial assistance, USDA regulations and the Administrative Manual, CRIS Instructions, C-REEMS instructions, and Agency records indicating receipt and approval of financial and technical reports. Portfolio review assessment outcomes from panels provide quantitative and qualitative assessments that focus and guide future efforts. Research projects supported with formula funds are reported in CRIS as per Administrative Manuals for Hatch, Evans-Allen, McIntire-Stennis & Animal Health Research Programs. For some extension formula programs there are proprietary database reporting systems.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: This program has made clear and consistent progress in achieving its long term performance goals. External Portfolio assessments demonstrate that that progress was made toward acheving 2004 program goals. CSREES uses its strategic plan, the GPRA process, and portfolio reviews to plan, execute, track and report on the progress of this program. The progrsm is working to increase animal and crop production efficiency, and to achieve an eight-fold in the market share if transportaion biofuels by 2010. The Index of Agricultural Productivity captures the growth in outputs not accounted for in growth of inputs. The Index (1996 =100) was 104.4 in 2002, the most recent year for which the Index was available. This is an increase from 103.6 in 2001. Agriculture's net value added to the US economy is forecast at $93 million in 2004, up from the 1999-2003 average of $89.2 billion. Bio-based transportation fule (ehtanol and biodiesel) use increased from 0.5% of total fuel (gasoline and diesel) use in 2002 to 1.9% in 2003. This is on schedule toward meeting the target of 4% of transporation fuel from biobased sources by 2010. Note: Even though these measures are under development, the answer received a "small extent" rather than a "no" since the agency was able to demonstrate past progress for the measures.

Evidence: Agriculutral productivity continues to grow (the Index was flat from 1996-1999). Index off Agricultural Productivity, www.ers.usda.gov/publications/agoutlook/AOTables/. Ball, V.E. and G.W. Norton, Ag. Productivity Measurement and Sources of Growth, www.cplpress.com/contents/C844.htm. Value Added by Ag. to the US Economy, www.ers.usda/publications'Agoutlook/AOTables/. Use of biofuels is increasing. Use of ethanol increased from 2.81 billion gallons in 2002 to 3.4 billion gallons in 2003. Biodiesel use increased from 0.5 million gallons in 1999 to 25 million gallons in 2002, to 30 million gallons in 2003 (USDA Office of Energy Policy and New Uses). The goal is an eight-fold increase in biofuel use. In 2003, the most recent year for which data is available, biofuels accounted for a 1.9% market share of road transportation fuel.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Targets for both annual performance and efficiency as defined were met or exceeded. Strateic Goal 1, Enhance Economic Opportunities for Agricultural Producers portfolio of work was assessed by six independent expert panels to determine progress toward achieving long term goals and solving targeted national problems. The aggregate (weighted by the relative funding of the six portfolios) was 80, and all individual portfolio scores were in the "exceeds expectations" range. Progress in two efficiency measures (proposal review costs and average time to award funding) continues to be made by improved use of information technology (e Gov and e Grants); however, a 28% increase in panel honoraria increased total costs per proposal and requires an upward adjustment of the benchmark. The adjustment also accurately reflects the salary and benefits of panel managers.

Evidence: Performance achievement is scored on expert assessment of PMA dimensions of relevance, quality and performance on a continuum that from "exceeds expectations" (67 to 100) to "meets expectations" (34 to 66) to "needs improvement" (0 to 33). The 2004 score (80), a dollar-weighted mean of the six portfolios meets the target. Quality, relevance and performance exceeded expectations (67) in all cases. Comparative benchmark data are not available prior to 2004. However, a meta-analysis of rate return estimates agriculture research and development (including extension) by the International Food Policy Research Institute in 2000 found that public funded rates exceed those of other sources (above average), www.ifpri.org/pubs/abstract/113/rr113toc.pdf. Agricultural productivity was demonstrated in 2002 to exceed that of other sectors of the U.S. economy. Ball, V.E. and G.W. Norton, Ag Productivity Measures and Sources of Growth (above average), www.cplpress.com/contents/C822.htm.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Efficiency and cost effectiveness are increased by reducing time-to-completion of proposal solicitation, review and funding, and by reducing peer vetting costs per proposal reviewed. The limit on recovery of indirect (F&A) costs for most CSREES competitively awarded grants has been increased to 20% of total Federal funds provided under each award. Transition to eGov and full electronic solicitation and submission of proposals will be completed in 2005.

Evidence: Mean time to completion of proposal review and award to date decline by 16 days over the 2003 level . Peer review costs were reduced nearly $50 per proposal between 2003 and 2003 and are forecast to deline another $15 in 2004.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: CSREES is the only extramural research agency in USDA. There is no comparable research entitiy in Federal, State, local government, or in the private sector that address agriculture and food issues at the national level.

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: CSREES Goal 1 portfolios and their component programs are evaluated by independent experts in the fields of science and by stakeholders. All Goal 1 portfolios were independently assessed by panels of outside experts this year. Assessments provided comments to help NPLs and implementing partners generate meaningful outcomes, in addition to scores of the quality, relevance and performance of each portfolio. Research affects productivity with a time lag; benefit streams typically extend beyond 30 yrs. Evaluative studies found rates of return of 40% to 60% for public R&D investment regardless of level of aggregation or geographical area. Extension reduces the time lag; its major effect on productivity is the short to intermediate period, although major technology adoption throughout the sector is a long term accomplishment due to the dynamic and constantly changing technology packages and management methods available. Economists estimate rates of return to extension from 20% to over 100%.

Evidence: Agriculture operates at high efficiency because of knowledge, techniques and technologies developed at land grant universities. Battelle's 2003 evaluation of Ohio State's Agricultural Research & Development Center, partly funded by CSREES, identified improved soil fertility; managed & controlled ag. pests; prevented, diagnosed, treated animal infectious diseases; controlled crop losses caused by pathogens; increased yields through breeding & enhanced cultivars; increased meat yields by enhanced breeding. Batelle found soybeans produced through OARDC discoveries annually generate $191 mil. economic output, $67 mil. income, 4,030 jobs www.oardc.ohio-state.edu/bab.pdf. Current research in Journal of Extension evaluated differences in users and non-users of extension in dealing with wheat diseases; CES programs and literature rapidly alerts farmers to services available regarding disease identification and prevention www.joe.org/joe/2004april/ent.shtml#a8.