Detailed Information on the
Grants for Nutrition and Health Assessment

Improving the focus of grant recipient reporting on outcomes.

Increasing planning and coordination with the Agricultural Research Service regarding the collection of stakeholder input.

Continuing to use competitive, peer reviewed grants, and proposing that no funding be provided for unrequested add-ons (earmarks).

Using "Grants.gov" (a web based peer review system), as well as virtual panels to improve the efficency of the grant review process, when appropriate.

Ensuring that all interested parties have the necessary access to grant information, as well as continuing to emphasize grant capacity building as appropriate.

Measure: Portfolio Review Score - Goal 4

Explanation:Improvements in the portfolio review score does not in itself ensure improvements in overall nutritional well being. There are intervening factors that are beyond the control of the research. Therefore, the use of an output measure in this instance is appropriate. Goal performance is externally assessed on a 5 year basis to determine progress toward solving targeted national problems related to each Strategic Objective. The score reflects expert assessment on 3 PMA dimensions of relevance, quality and performance, along a continuum from exceeds expectations, to meets expectations, to needs improvement. The reported score is a mean of quantitative scores for the Goal's objectives.

Measure: Dietary improvement achieved by EFNEP participants

Explanation:The percentage of EFNEP graduates who improve their diets toward meeting MyPyramid recommendations following their participation in EFNEP. Past data from 1993 - 2005 showed 93% ate nearer to Food Guide Pyramid recommendations. The MyPyramid criteria have been incorporated into the educational program and the evaluation system. The goal will be to maintain this high level of improved diet intake behavior.

Measure: Development and use of effective intervention methods and strategies to change behavior and improve diet and physical activity in target populations.

Explanation:The development of effective intervention methods and strategies does not in itself ensure improvements in overall nutritional well being. There are intervening factors that are beyond the control of the research. Therefore, the use of an output measure in this instance is appropriate. These new interventions lead to advances in the ability of educators to improve results for dietary intakes and food related behavior for target audiences. New interventions reflect the successful application of basic nutrient, psychosocial and educational research. The goal will be to add one new intervention per year.

Measure: Internal Portfolio Review Score

Explanation:Goal performance is internally assessed annually by NPLs to determine progress toward solving targeted national problems related to the Objective. The score reflects expert assessment on 3 PMA dimensions of relevance, quality and performance, along a continuum from exceeds expectations, to meets expectations, to needs improvement. The reported score is a mean of quantitative scores for the Goal's objectives. The baseline of 86 in 2006 is the Portfolio Review Score as determind by the external panel (above).

Measure: Cumulative Dollars Saved for Grant Review

Explanation:Dollars saved reflects the average salary saved by calculating the number of calendar days saved annually between receipt of proposal and date funding awarded for competitively reviewed proposals, then multiplied by the average daily salary for CSREES employees. The annual savings can be derived be comparing the change in cumulative savings between two consecutive years.

Measure: Proposal Review Time in Days.

Explanation:Average number of calendar days between receipt of proposal and funding date funing awarded for competitively reviewed proposals. Average is calculated by summing the total number of days for all major competitively reviewed proposals and dividing by their total number.

Is the program purpose clear?

Explanation: The purpose of the Nutrition Portfolio is to lead & support extramural research, ed & Ext programs in the land grant system & with other entities to improve the nutrition & health of Americans, esp those who are most vulnerable. Research addresses: the requirements & function of nutrients (used as a basis for Federal nutrition policy); the psychosocial & environmental factors that influence food-related behavior; the development of effective strategies to improve diet & physical activity & the development of evaluation systems to determine the effectiveness of interventions. CSREES supported & other pertinent research is translated into science-based, culturally relevant outreach programs for audiences targeted through EFNEP, 4-H, Ext nutrition & health programs, FSNE & the Community Foods Program (improving access to food in local communities). CSREES research & Ext programs are used as a resource & model for other programs throughout the country

Evidence: The research, ed & Ext work of CSREES' Nutrition Portfolio is clearly defined by legislation & made explicit in CSREES' Mission & Strategic Plan (2004-09). One of the USDA & CSREES Strategic Goals is to "Improve the Nation's Nutrition & Health". The work also supports the Presidents HealthierUS Initiative. Responsibilities are authorized through 14 funding lines as legislated by Congress. The Hatch Act, Smith-Lever, Evans-Allen Program, & Congressional earmarks; the National Research Initiative (NRI) Competitive Grants, Section 2(b), Act of 1965, 8 U.S.C. 450i(c); & Special Research Grants, Section 2(c), Act of 1965, 7 U.S.C. 450i(c) are a few examples that relate specifically to work in this Portfolio. There is an emphasis placed on diversity & minority serving institutions, including 1994, 1890, etc.

Does the program address a specific and existing problem, interest, or need?

Explanation: Diet links to 4 of the 10 leading causes of death. Dietary Guidelines for Americans are established by USDA & HHS. Despite these guidelines, Americans consume less than the recommended intake of calcium, potassium, fiber, magnesium & vitamin E, but too many calories & too much saturated & trans fats, cholesterol, sugar & salt. USDAs Healthy Eating Index showed only 10% of Americans had a good diet in 1999-2000. Imbalanced caloric intake & physical inactivity causes overweight & obesity. In 1999-2000, 65% of U.S. adults were estimated to be overweight or obese (a prevalence 16% higher than age-adjusted data from 1988-94). More alarming, 16% of the children & adolescents were overweight, a 45% increase from the estimates in 1988-94. Obesity is now an epidemic. Poor diet links to 4 of the 10 leading causes of death & obesity is now considered an epidemic. CSREES' Nutrition Portfolio addresses these problems. The research component of CSREES' Nutrition Portfolio provides a basis for recommendations on what Americans should eat. Research supported by CSREES & others shows the large discrepancy between recommendations & the current American diet & level of physical activity. The Ext component of CSREES' Nutrition Portfolio provides the national programs (EFNEP, FSNE, Ext nutrition & health programs) to guide & motivate people, esp the most vulnerable, from where they are to where they should be.

Evidence: The most comprehensive synthesis of current data on nutrient needs & the American diet was released by the Dietary Guidelines Advisory Committee www.health.gov/dietaryguidelines/dga2005/report. This report showed that Americans consume less than recommended intakes of calcium, potassium, fiber, magnesium & vitamin E, but too many calories & too much saturated & trans fats, cholesterol, sugar & salt. This report was also the basis for The Dietary Guidelines for Americans http://www.usda.gov/cnpp/DG2005/index.html jointly released by USDA & HHS as a statement of Federal nutrition policy. USDA's Healthy Eating Index showed only 10% of Americans had a "good" diet in 1999-2000 http://www.usda.gov/cnpp/healthyeating.html. The National Health & Nutrition Examination Survey estimated 65% of U.S. adults & 16% of adolescents to be overweight or obese in 1999-2000 www.cdc.gov/nccdphp/dnpa/obesity/trend/index.htm

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: As discussed in the evidence, project proposals are reviewed to ensure that there is no duplication. The most comprehensive synthesis of current data on nutrient needs & the American diet was released by the Dietary Guidelines Advisory Committee www.health.gov/dietaryguidelines/dga2005/report. This report showed that Americans consume less than recommended intakes of calcium, potassium, fiber, magnesium & vitamin E, but too many calories & too much saturated & trans fats, cholesterol, sugar & salt. This report was also the basis for The Dietary Guidelines for Americans http://www.usda.gov/cnpp/DG2005/index.html jointly released by USDA & HHS as a statement of Federal nutrition policy. USDA's Healthy Eating Index showed only 10% of Americans had a "good" diet in 1999-2000 http://www.usda.gov/cnpp/healthyeating.html. The National Health & Nutrition Examination Survey estimated 65% of U.S. adults & 16% of adolescents to be overweight or obese in 1999-2000 www.cdc.gov/nccdphp/dnpa/obesity/trend/index.htm

Evidence: The most important mechanism for ensuring that CSREES' nutrition research & Ext agenda addresses the most important issues & audiences without redundancy/duplication is through stakeholder input from professionals & practitioners at the Fed, state & local level who have a broad knowledge of the existing nutrition programs. For competitive research, NRI peer review panels familiar with all relevant research select proposals addressing significant gaps (funding is available for only the top 15% of proposals). Proposals for noncompetitive research are reviewed at the university (& regional level for multistate projects) & Federal (NPL) level. These proposals must provide evidence based on a review of the Current Research Information System (CRIS) that there is no duplication among similarly funded projects. No duplication/redundancy of nutrition Ext programs is ensured by several mechanisms of coordination & communication at the Federal, state & local level (see below).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: A comprehensive review of the Nutrition Portfolio by an outside, independent panel assessed it to be of high quality & therefore free of major flaws. Earmarks & formula-funded projects do not detract from CSREES' ability to design a high quality program. All proposals (competitive, formula, or earmarks) are held to the same high standards. From the earliest development stages, NPLs work with formula-funded & earmark projects to ensure they address administration priorities & are of the same quality as highly ranked competitive proposals. Research proposals are reviewed by an outside, independent panel &/or by one or more CSREES NPLs to ensure quality. EFNEP has a strong continuous evaluation system that assures a program design of high quality, free of major flaws. For other Ext programs, annual Plans of Work are reviewed by NPLs for the same purpose. No funds for any research or Ext programs are released until quality & freedom from flaws are assured. While earmarks are evidence of a basic design flaw of the program, in this instance they represent only about 10% of the total program.

Evidence: CSREES' Awards Management Branch & NPLs with subject matter expertise effect a rigorous review process for all types of research proposals (competitive, formula & earmarks) to ensure high quality & freedom from major flaws. Through the NRI, funding is available for only the very best proposals (top15%). Annual progress reports from all types of research projects are required to be submitted to CSREES' Current Research Information System (CRIS) which is open to the public. If a project has not remained flaw free or made adequate progress, funds are withheld. A new on-line State POW system will track Ext programs to ensure quality. EFNEP annual & multi-year summaries are shared with States to highlight data trends. Recommendations for improvement are included to ensure continuous high quality. Cost-benefit analyses showed a $10+ savings in health care per $1 for EFNEP implementation.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The Nutrition Portfolio is targeted to the administration priority of improving the Nation's nutrition & health. Research program beneficiaries are policy makers, program developers & other researchers. Ext program beneficiaries are program participants. CSREES' research & Ext programs address the most important nutrition issues for the most vulnerable population groups. Only research proposals that can make the most significant contribution are funded. Results are shared with beneficiaries through publications, presentations, & websites. The 2 largest intervention programs carried out by Ext, EFNEP & FSNE, are targeted to low income youth & families, who have a high risk for chronic disease, obesity & poor health. Among the EFNEP clients, 69% are non-white & 78% receive 1 or more Federal Food Assistance programs at entry & 9% more gain access to food assistance through EFNEP. This indicates that the targeted beneficiaries are receiving help.

Evidence: Agency resources are targeted based on administrative priorities, the external REE Advisory Committee, stakeholder input & needs identified in numerous reports, such as the 2005 Dietary Guidelines Advisory Committee Report, the Institute of Medicine reports on Dietary Reference Intakes & its 2005 report "Preventing Childhood Obesity". USDA's nationwide food intake surveys show the gap between recommended intakes & actual intakes by Americans, especially among low-income & minority groups. These are the groups targeted by the EFNEP, FSNE, & other Ext programs. Research productivity as indicated by publications, presentations, & websites, & evaluations of Ext programs such as impact data from EFNEP (www.csrees.usda.gov/nea/food/efnep/impacts.html) show targets are being met - 93% of EFNEP graduates improve their diets.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: As part of CSREES' balanced strategy to support improvement of the Nation's Nutrition & Health, the Nutrition Portfolio has 3 quantifiable long-term outcome-based performance measures. The 1st uses expert reviews scores to track relevance, performance, and quality of 100% of the portfolio. The 2nd focuses on the effectiveness of CSREES flagship nutrition edu. intervention program EFNEP. Health disparities, obesity & chronic disease are all impacted by poor quality diets, thus EFNEP's focus is on changing behavior to more closely match the Dietary Guidelines & MyPyramid recommendations. Because EFNEP is the model for much of the larger FSNE program, it is indicative of the success of that program as well. The 3rd focuses on the dev. & use of new interventions that enhance the ability of educators to improve American diets, physical activity & related behav. This measure is indicative of the success of research on nutrient needs & function, food choice related behaviors & interventions

Evidence: The Nutrition Portfolios have three specific long-term measures that focus on outcomes. (1) The broadest long-term performance measure established is the five-year external portfolio review score. The baseline target set for the 2006 review was 75. (2) The percentage of EFNEP graduates who improve their diets toward meeting the Food Guide Pyramid recommendations has been 93%. The measure will be changed to the new MyPyramid recommendation, with the intention of maintaining this high level of improved dietary intake behavior; (3) the development and use of effective intervention methods and strategies to change behavior and improve diet and physical activity in target populations. The baseline for 2005 is one intervention program

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The recently completed independent expert panel review of the Nutrition Portfolio found existing long-term targets to be ambitious. Because of the high score given for the portfolio, continuous improvement in overall relevance, quality and performance will be a challenge. EFNEP is an intensive educational intervention. The development and use of effective intervention methods and strategies to change behavior and improve diet and physical activity in target populations will be a special challenge because biological & behavioral research is needed as a basis for interventions and behavior is notoriously difficult to change. Changing behavior to improve diet and physical activity is notoriously difficult.

Evidence: The outside, expert panel that recently reviewed the Nutrition Portfolio gave it an overall high score of 86. Given that this score is near-optimal, the goal of improving by 2 points per year is ambitious. The percentage of EFNEP graduates who improve their diets toward meeting the MyPyramid recommendations has been shown to average 93 percent. While the targets do not increase above 93%, that is considered to be an extremely high level of performance. Increasing that level of performance above 93 percent within available funding, which could be accomplished by providing more intense service to participants, would result in the program being able to providing services to fewer participants.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: CSREES has 3 annual performance measures to track progress toward long-term goals. Annual internal assessment of the Portfolio is used to track progress toward achieving the long term measure, which is the target external Portfolio Review Score. Each year, NPLs review and self-score the activities & results of the Portfolio regarding relevance, quality & performance, which includes collecting data to assess progress toward the long-term target. The NPLs report on steps taken to address recommendations made by the external review panel. Thus, if the annual targets are met, the long-term target will be met as the final annual increment in the progression. Since the annual review incorporates specific program recommendations, as the internal scores improve, it is more likely that the other two long-term outcome measures will show steady progress toward their long-term target. Two efficiency measures have been developed to measure the average cost/proposal reviewed & the average days to review a proposal and award funding. Achieving these measures contributes to long-term outcomes by helping to keep CSREES' award processes efficient.

Evidence: Baselines and targets through 2010 have been established for all measures and data have been collected for 2005 and are beginning to be collected for 2006. The external panel that assigned the Portfolio Review Score provided recommendations that are used by NPLs in their portfolio management to make ongoing improvements to the portfolio, which will be reviewed annually in the self-assessment. Thus, the three annual measures (the internal portfolio review score; the average cost for proposal reviewed; and the average number of days to review a proposal and award funding) collectively ensure that portfolio management makes changes as needed for accountability, to improve programming, and to meet the long-term targets.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: To achieve the ambitious LT goals set for the Nutrition Portfolio, ambitious targets have been set for annual measures. The annual measure of the overall Nutrition Portfolio, as assessed internally, must improve by 1 point per year from the base of 86 in 2006. This annual assessment includes annual measurement of the percentage of EFNEP graduates improving their diet (which must be maintained at 93%) & measurement of the development & use of effective intervention strategy (which must be increased by one annually). Ambitious annual targets have been established for the efficiency measures to continuously reduce the cost & time for reviewing proposals.

Evidence: An external, expert review panel gave the Nutrition Portfolio an overall 5 year score of 86 in 2006. This was established as the baseline for the annual internal evaluation. The baselines for the efficiency measures were set at an average cost of $530 per proposal & an average review time of 220 days per proposal in 2004. The cost per proposal measure reflects transportation, lodging costs & other variables beyond the control of the Agency, honoraria (which must be competitive), & the increased use of more effective but more costly on-site, rather than ad hoc, reviewers; annual targets call for holding review & processing costs to a real increase of 2%, & an average 2% reduction in review time. The review time per proposal reflects the issues that relate to transportation & communication time. Hurricane Katrina had an adverse effect on the review time in 2005. In the future, the move to electronic submissions & review should improve efficiency.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: CSREES enters into education, research, and/or extension partnerships only when mutual benefit is clearly demonstrated & it is relevant and consistent with CSREES mission, Strategic Plan & long term & annual goals. All submitted proposals & plans of work specify the scope of work, milestones, measures, outputs & outcomes, along with performance monitoring procedures. Submission of proposals & acceptance of awarded funds requires commitment by grantees & partners to work to accomplish annual & long term goals, progress on which must be reported to CSREES on an annual basis.

Evidence: CSREES reviews 5000 proposals & funds 2000 proposals, cooperative agreements & other transactions yearly. All proposals & plans of work must include measures relevant to CSREES long term & annual performance goals & targets. NPL's receive annual progress reports used to review & monitor performance to assure continued commitment of partners. Funds are withheld or deferred if the commitment is not maintained. NPLs work with peers to develop RFA guidance to explicitly define long term & annual goals. grants, cooperative agreements, & some formula funds, these progress reports are collected in the CRIS reporting system (cris.csrees.usda.gov). See Award Terms & Conditions http://www.csrees.usda.gov/business/awards/awardterms.html. Hatch, Smith-Lever, Evans-Allen, and 1890 Extension funds must report on progress against their POW annually and these reports must be approved before funds are released

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: All Nutrition Portfolios are independently assessed & evaluated by external panels of experts every 5 years, with a 2006 baseline score. Outcomes are also assessed internally on an annual basis. Other external reviews, evaluation studies, analyses of CSREES programs, & agency's self-review documents are presented as evidence to the review panels. Portfolios are rated on OMB's criteria of relevance, quality and performance. Both of these assessments provide useful comments & recommendations to help NPLs set goals for improving the Portfolio. To ensure quality panel members are scrutinized for any bias, conflicting interests, or other issues related to independence. The panels address the full scope of nutrition related research, education and extension work down to the level of Knowledge Areas and contributing programs. Having external panels meet on a 5-year basis is common practice in research and allows sufficient time for advances to take place.

Evidence: The results of the independent expert portfolio review evaluation for the Nutrition Portfolio, conducted in February 2006, is documented in self-study reports that can be found in the Long Term Measures tab. The Nutrition Portfolios received an independent review score of 86 which "exceed expectations." To ensure quality, the "Guidebook for Preparation for Portfolio Review Expert Panel (PREP)" was developed for the NPLs to use in preparing external review materials. A description of the PREP process can be found on the web at http://www.csrees.usda.gov/business/reporting/portfolios.html.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget & Performance Integration (BPI) directly links long range and annual goals as defined in the annual Departmental Performance Plan, the REE Quarterly R&D performance reports and three other Strategic Plans submitted to the Under Secretary. Explicit knowledge areas defining the program are exclusive and non-duplicative, thus directly linking budget lines with goals, objectives, the program and related projects in a transparent manner. CSREES works with USDA s Office of Budget and Program Administration (OBPA), which provides direction and administration of USDA's budgetary functions including development, presentation, and administration of the budget and analyzes program and resource issues and alternatives. The CSREES 2004-2009 Strategic Plan identifies key policy and management objectives integrated within CSREES budget priorities and, at the same time, provides accountability through a series of annual performance plans. http://www.csrees.usda.gov/about/strat_plan.html In addition, the Budget Explanatory Notes (Congressional justiication) includes both direct and indirect costs associated with the agency program goals.

Evidence: Annual & long-term performance goals & accomplishments were an explicit component of the FY2006 President's Budget Request (see Agency budget requests, Explanatory Notes, tables, exhibits, & accompanying information). The program aligns specific projects by knowledge area and funding line with the objectives of the CSREES & USDA strategic plans. Using CRIS now and the State Plan Of Work (POW) System beginning sometime this year, NPLs can track the impacts of projects by knowledge area & by funding source. http://cris.csrees.usda.gov/star/system.html. The new electronic State POW System released this year as part of the CSREES OneSolution data & reporting system currently under development will continue to strengthen the transparent links between agency budget & performance measures. Planning & Accountability is working to strengthen the links between budget justification & budget & performance integration submissions. http://www.csrees.usda.gov/about/offices/planning.html.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Ongoing strategic planning efforts were undertaken to fully align the CSREES Strategic Plan with the REE & USDA Strategic Plans. Comprehensive Knowledge Area codes developed for all agency work align CSREES higher education, research, & extension portfolios with the USDA strategic objectives. Budget Performance Integration (BPI) ties the Agency's budget request preparation, justification, & reporting to needs, strategic plans, & performance measures. State POWs require 5-year strategic planning efforts including extensive stakeholder engagement, as directed by AREERA. POWs link funding to projected outcomes, and indicators used to measure progress. These plans are revised annually, creating rolling POWs always extending five years into the future. NPLs reviews of LGUs are timed to coincide with their strategic planning efforts. All plans for nutrition research, ed & Ext. programs must be consistent with the Strategic Goals & progress & accomplishments are measured against these goals.

Evidence: CSREES demonstrates comprehensive linkages of Nutrition Portfolio between congressional authorizations & appropriations, the Strategic Plans, the Departmental & Agency mission, the Departmental Performance Plan, & specific areas of science & education. Linkages of all research projects with Strategic Goals are shown on the CRIS website. All three of the strategic plans are posted together on the agency web site at www.csrees.usda.gov/about/strat_plan.html. The CSREES OneSolution data management & reporting system comprehensively links strategic planning, Budget & Performance Integration (BPI) & output, outcome & impact reporting.

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: Stakeholders are involved in planning, development & review. NPLs consult with researchers & educators, community groups, USDA advisory groups & industry to assure CSREES projects complement programs in other institutions. Coordination with other Federal agencies is very strong with several coordinating committees. Their responsibilities include discussions of plans, review of RFAs & development of collaborative efforts. External portfolio reviews enlist outside experts to examine CSREES programs & benefits & monitor progress toward solving national problems & to make recommendations for future direction. Panels of outside independent reviewers review projects applying for competitive funding, & rank projects high priority when they are of high quality & are expected to yield the most value. External & internal analyses usually agree; if not, NPLs address external concerns before funding.

Evidence: CSREES designs programs in collaboration with other agencies (e.g., ARS, ERS, CDC, DOE, DOD, HHS, NIH, NSF) to focus on its mission & resources, & its partnership with the university system. CSREES leaders confer quarterly with REE agencies (see REE Quarterly R&D Perf. Mgmt. Plan & Quarterly Reports). CSREES' integrated mission is an asset, adding value to efforts of other federal org. & the private sector. CSREES participates in USDA & HHS Nutrition Coordinating Committees which have representation from each internal agency with a role in nutrition. USDA & HHS jointly review all dietary guidance materials for consistency with Federal nutrition policy. Several coordinating committees focus specifically on obesity. The highest level committee includes USDA, HHS, OSTP, & NSF. Portfolio reviewers found programs compared well with similar programs. They found them to have high sig. of findings, high alignment with current state of science; & moderate productivity. (See 2006 Panel Reports)

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The Agency develops annual budget and programmatic priorities based on Administration priorities, existing & emerging needs, recent successes in research, the state of science, educational capacity, and direct input from partners, constituents & other providers. CSREES secures inputs from stakeholders through both formal and informal processes. All RFAs invite comment. Annual meetings for project directors include input on future directions, grantsmanship workshops & professional meetings also invite comment. Input from the Committees on Organization & Policy of Extension (ECOP) & Experiment Stations (ESCOP) & the 4 regional associations of AES provide direction on future priorities. The comprehensive BPI process makes the linkages and potential gaps & overlaps apparent as the budget request is being prepared. This allows for the refinement of the budget request, strengthened justifications, and appropriate program changes before the budget request is submitted.

Evidence: CSREES NPLs collaborate on the development of Request For Applications (RFA), including the prioritization, the science & education, and the integration of portfolio work in the agency. Draft RFAs are shared with the USDA Nutrition Coordinating Committees for their input & to coordinate planning. Comprehensive NRI RFA Planning Worksheets are prepared for each priority issue, clearly defining mission-relevant problems to address in short, intermediate & long-term goals as a result of input from stakeholders. Ag. Research, Education & Extension Reform Act (AREERA) requires recipients of formula funds to collect stakeholder inputs yearly. They must document this in their POW. Additionally, CSREES holds workshops with constituents on a regular basis that are designed to help prioritize. This information, from these sources along with Portfolio Review recommendations is integrated in the BPI process in collaboration with OBPA & OMB to guide Agency budget requests & funding decisions.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Recipients of competitive & line item grants must submit annual progress & final reports through the CRIS. NPLs use CRIS data to monitor progress, make recommendations, & make changes in priorities, focus & program admin. CSREES collects info on progress of research formula awards through CRIS; research & extension activities through State POW reporting - with NPL oversight; & extension work through POW & annual reports. Extension s online State POW System will better integrate performance data across edu, research & extension. OneSolution data & reporting system will further inc timeliness & quality of data/info, reduce reporting burdens on key program partners, & inc program performance mgmt. The EFNEP evaluation sys operates at the county level to provide program mgmt data case loads, effectiveness, feedback to clients etc, & send aggregated data to states for review, mgmt & compilation into state summaries for the federal office assuring timeliness & increased accuracy of data.

Evidence: Terms & Conditions for competitive & line item awards specify annual & final report requirements. Reports determine progress of grants & if additional funds should be provided. NPLs use info to compile reports, measure impacts & respond to inquiries from mgmt, public, & Congress www.cris.csrees.usda.gov. Formula funded research is reported in CRIS per Admin Manuals for Hatch, Evans-Allen, Mc-Stennis & Animal Health Research Programs. All 1862 & 1890 Institutions submit 5-yr POW & Annual Reports of Accomplishments & Results (Guidelines for State POW & Annual Report Guidelines). Reports measure impacts in addressing critical issues as identified by stakeholders. Multiple state teams developed the rationale & design of the NEERS5 system http://www.csrees.usda.gov/nea/food/efnep/neers5/neers5.html to be released 2006. OneSolution (estimated to save $20.2 mil. over 7 yrs with benefit cost ratio of 1.89), Business Case, Final Draft 3/3/05, CSREES Customer Service Satisfaction Standards.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: CSREES has 3 inter-related but indep. internal units (Program, Awards Mgmt. Branch (AMB) & Funds Mgmt. (FM)) that oversee the approval, admin oversight, & funds accountability. NPLs oversee projects & interact with grantees to ensure project performance & program results. NPL's have specific program management performance standards that are included in their annual evaluations.. AMB examines ea project to ensure eligibility, admin. completeness, & authorizes release of funds when standards are met. CSREES is subject to Financial Assistance regulations & grantees required to manage & expend funds according to award Terms & Conditions (based on OMB circulars, statutory requirements, USDA & CSREES regulations). Award recipients report grant expenditures via a PSC-272, Federal Cash Transactions Reports & SF-269 Financial Status Reports. Grantees submit progress & final reports & undergo annual audits per OMB Circular A-133.

Evidence: Budget Office develops Control Tables with NPL assignments for the fiscal year. The Office of Extramural Program (OEP) develops guidelines & training to assist NPLs in solicitation, review, & award of grants (e.g., RFAs, Award Terms & Conditions, Guidelines for State POWs). FM releases grant funds as authorized by the AMB & POW staff after programmatic review is complete & admin. requirements met. NPLs, AMB, & FM follow up with awardees to ensure required financial & technical reports are received & satisfactory progress is being made. 2005 performance standards for NPL, AMB, & FM staffs had cascading standards referring to CSREES strategic goals or the President s Mgmt. Agenda. Each yr the NPL & OEP staffs develop a Schedule of Awards with target dates to manage workload/evaluate Agency performance. See http://www.csrees.usda.gov/business/business.html for Grant guidelines & regs. Some states use EFNEP s reporting system to compare different strategies & audiences.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Funds are (unless No-Year or Two-Year) required to be obligated in the year appropriated, which is monitored through C-REEMS & FFIS reports. Annual reports, some of which are included in the Explanatory Notes, are prepared by OEP staff & reviewed for accuracy. Each year the Agency drafts a Schedule of Awards outlining target dates, peer panel dates, & award dates. OEP staff & NPLs are held accountable for these dates. Audits (e.g., A-133 & OIG), reviews (e.g., outreach & administrative) & mandatory reporting by partners (i.e., technical & financial), along with active post-award management, ensure that funds are spent for the intended purposes. OEP has developed an internal Formula Distribution Handbook for Staff to follow. CSREES has developed a new POW Reporting System for formula funds for CSREES administrative & program managers with set standards & requirements. Award recipients must expend funds to achieve only funded objectives & according to Award Terms & Conditions.

Evidence: CSREES prepares a schedule of obligations, monitors unobligated balances, compares obligations with scheduled estimates, & takes appropriate action. The dispersal of obligated funds can be deferred if deficiencies &/or deviations are detected &/or administrative requirements are not met. Awardees are required to take proactive corrective action in order to secure the release of funds. Agency personnel periodically review the Status of Funds Report & other management reports to monitor the availability of grant funds. Post-award oversight conducted by CSREES includes follow-up for any outstanding A-133 audits & Financial Status Reports, (required upon termination of award & annually for formula funds). Staff perform outreach & administrative reviews as well as Improper Payment Information Act (IPIA) transaction tests. Awardees are required to spend funds only on funded objectives, follow budgetary guidelines & reporting requirements, & fully expend all funds within a finite period.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: CSREES has an internal grants management system, CREEMS, that assists management & staff by providing budget, expenditures & statistical data related to the administration of grants (e.g., time to process a proposal from receipt to award; time to process a post-award action). The system serves as a management tool for tracking action items & administrative compliance requirements. The system is CSREES' innovation in integrating all grant processes (programmatic, administrative, & fiscal) in one database. Annual efficiency measures & targets compare proposal review time & costs for continual improvement. The OneSolution data & reporting system procedures will increase the timeliness & quality of performance related data & information, reduce the reporting burden on key program partners, & increase program performance management. It is estimated to save CSREES $20.2 mil. over 7 years, with a benefit cost ratio of 1.89. The program has two annual efficiency measures, with baselines and annual targets. They are the average time it takes to review a grant proposal and the average cost to review a grant proposal.

Evidence: See the Annual Efficiency Measures tab. For CREEMS documentation, see www.csrees.usda.gov/business/other_links/egov/csrees/creems.html. The Agency has a comprehensive IT Strategic Plan to increase the use of information technology to increase cost effectiveness & time efficiency, see www.csrees.usda.gov/about/offices/pdfs/strategic_plan_03-08_Final.pdf. See also CSREES Customer Service Satisfaction Standards. The Agency also has a specific Information Technology Strategic Plan, revised July 2004, www.csrees.usda.gov/about/offices/pdfs/strategic_plan.pdf; for OneSolution documentation & analysis of projected benefits, see OneSolution Initiative, Business Case, Final Draft March 3, 2005. The PACE and eCRU Committees provide forums for program and administrative staffs to discuss Agency performance standards, processes to attain standards, and standard evaluation.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The portfolios are highly integrated & coordinated within the agency & externally. CSREES constituents, stakeholders, and partners are routinely involved in scientific research, education & extension program planning, development, and review. Ongoing coordination occurs within USDA, with federal interagency working groups, and with other partners. NPLs collaborate with scientists and educators within/beyond the Department and with industry, producers and citizens to maintain the relevancy of programs, to avoid redundancy with other programs, and to add value and extend benefits of CSREES and other agency program efforts. NPLs and their counterparts also collaborate on the development of RFAs & guidance for Plan of Work, including priority areas. Collaboration and coordination between EFNEP and other programs reaching low income families is critical to assure that these efforts are complementary and not duplicative.

Evidence: CSREES constituents, stakeholders & partners are involved in program planning, dev. & review. NPLs collaborate with scientists & educators & with an advisory group to maintain the relevancy of edu., research & extension & avoid redundancy. CSREES connects with USDA & HHS work through 2 Nutrition Coordinating Committees & through the Dietary Guidance working Group. CSREES is connected with the Institute of Medicine (membership on advisory committees) & professional orgs (service on various committees). CSREES supports the Asso of Nutrition Departments & Programs including Nutrition & Food Science dept chairs from land-grant and non land-grant universities. The Cooperative Extension System is the most important network for developing educational & environmental interventions to help people improve diet & physical activity. EFNEP works with >1500 WIC & Food Stamp offices, has 6,000+ agreements & participates on 1320 state & local coalitions to provide consistent messages & outreach.

Does the program use strong financial management practices?

Explanation: All grantees must comply with Federal assistance & implementing regulations. Audits, reviews & mandatory reporting, with active post-award mgmt., ensure that funds are spent for agreed upon purposes. Grant recipients are required to report expenditures each quarter 45 days after close of the quarter via PSC-272 Federal Cash Transactions Report, through DHHS-PMS. Draw down requests are denied for failure to submit this report. If an institution fails to submit technical & financial reports according to predetermined due dates, the formula funds are withheld until all requirements have been satisfied. Funds also are placed on hold for competitive awards until budget & admin. requirements are met. Each year selected transactions are reviewed by the USDA OIG (i.e., financial statement audit). CSREES is working with the USDA Office of the Chief Financial Officer on the A-123 requirements for which CSREES will have to certify that its internal controls are adequate & functioning.

Evidence: CSREES has received unqualified financial statement audit opinions & has not reported any material weaknesses since FY 2002. For the FY 2006 IPIA cycle, CSREES performed 6 site visits with no major findings. CSREES via the eGrants Exec Board continues to support improvements in the Funds Management Module of CREEMS (i.e., enhanced reporting & more automation). Program, administrative & financial management is conducted per OMB Circulars for Federal assistance, USDA regulations & the Administrative Manuals, CRIS & CREEMS instructions, & Agency records indicating the receipt & approval of financial & technical reports (i.e., receipt & review of financial & technical reports) for all 3 funding mechanisms: competitive, line-item, & formula funds. See Award Terms & Conditions http://www.csrees.usda.gov/business/awards/awardterms.html Terms & Conditions.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: CSREES has implemented steps to address accountability issues & initiatives through OEP, which provides oversight to OIG audits & hotline complaints, GAO evaluations, IPIA & FMFIA compliance, A-123 requirements, & liaises with Planning & Accountability for POW. Concerns about the POW process are being addressed through a committee. The A-123 process is expected to result in improvements. As a result of the Customer Satisfaction Survey (CSS) (Spring 2005), 3 committees have been formed with state partners (i.e., administrators, applicants, & business officers) to follow up on the Survey. Draft recommendations were made in May 2006. A Final Report will be presented in July 2006. The Deputy Adm is responsible for the mgmt of the portfolio. CSREES staff participates in various intra-agency & interagency committees that address streamlining & improving business processes (e.g., PACE and eCRU Committees).

Evidence: Transition to e-Gov will result in paperless proposal & plan submission, review, approval & funding authorization to reduce processing time & improve accountability. CSREES plans to require electronic submission for all NRI and SBIR grants as well as provide the option for all programs in FY 2007. Guidelines for review & approval of formula funding have been implemented. The 3 CSS committees have been meeting & have drafted recommendations for the July 2006 report. PACE and eCRU committees continue to meet, discuss & implement initiatives to improve CSREES business processes & enhance program & fiscal accountability. Program Administration Coordination Effort (PACE) has completed Service Standards that set temporal guidelines for planning, awards management & time-critical administrative & management activities as policy recommendations for strategic & short-term planning, budget justification, performance measurement & evaluation, contributing to review & improvement of administrative functions & assisting NPLs assess accomplishments & determine changes to improve performance & efficiency.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: AREERA requires a Plan of Work & an Annual Report to be submitted for every formula funded activity. Funds are not released until proposals & plans of work are approved by NPLs. Grantees annually report on their progress in achieving stated goals for each funded activity. These are reviewed by NPLs to ensure that sufficient progress is being made. Proposals & accomplishment reports are entered in the agency's CRIS reporting system. To ensure success and fulfill requirements, CSREES can place additional controls on new grant recipients. Oversight of grantees is also made through OIG audits, OIG hotline complaints, GAO evaluations, IPIA & FMFIA compliance, A-123 requirements, review of financial status reports, A-133 audits, & agency site visits to verify program, grant, & financial compliance.

Evidence: In accordance with section 103(e) of AREERA a merit review process is required by every 1862, 1890, and 1994 land-grant institution to obtain formula funds under section 3(b)(1) and (2) of the Smith Lever Act, and formula funds for research programs under sections 1444 and 1445 of NARETPA & section 3(c) and (2) of the Hatch Act of 1987 (Regular Formula Funds). Furthermore, a scientific peer review process is required for programs funded under section 3(c)(3)of the Hatch Act (Multistate Research Funds). Site visits, audits & dept. program reviews are conducted by NPLs on an annual basis. More recently the new role of "State Liaison" has been created in which NPLs are assigned to coordinate the activities of cooperating institutions within one or two states. In addition, CSREES organizes an Admin. Officers' Conference & other Agency forums for admin. & programmatic mgmt. of CSREES grant funds http://www.csrees.usda.gov/business/reporting/planrept/additionalguidance105204.html.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: CSREES annually collects, compiles and disseminates grantee performance data down to the funded project level, and makes data publicly available in a transparent manner via the searchable CRIS Web Site at www.cris.csrees.usda.gov. Grantees are contractually required to submit Annual Reports of Accomplishments and Termination Reports to describe their performance in achieving their stated goals as well as to describe their progress on funded projects. Reports include references to publications and websites. In addition success stories from extension projects, especially EFNEP, are published annually along with annual reports or statistics from programs. Some stories are picked up by CSREES communications office and prepared for media dissemination.

Evidence: The widely used CRIS is a comprehensive repository of performance data for funded projects from CSREES, & for work of related agencies with similar missions & activities. These are identified by knowledge area, subject of investigation, & field of science, & include proposal summaries, annual performance reports, outcomes & impacts, & termination reports. This information is searchable, available to the public, & has the capacity to produce various types of reports according to customer needs. See www.csrees.usda.gov/business/other_links/egov/csrees/cris.html. Another CSREES innovation, the Competitive Research, Education, & Extension Management System (C-REEMS), makes comprehensive information on proposals, their review & award management available to NPLs in creating specialized reports. See www.csrees.usda.gov/business/other_links/egov/csrees/creems.html

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Approximately 90% of the funding for this portfolio is awarded on a competitive basis. An extensive, rigorous competitive review process including a panel manager, qualified reviewers, strict guidelines & safeguards to avoid conflict of interest is in place for all competitively awarded grants. Panels review proposals for their relevance, scientific quality, appropriate personnel, facilities & budgets & the likelihood of success. In all cases, release of funds is deferred pending Institutional Review Board approval for studies involving animals or human subjects. CSREES works with new and first-time recipients to ensure they understand & adhere to Federal Financial regulations and policies. CSREES conducts outreach to encourage new grantees through presentations at professional societies and other stakeholder meetings, and through annual grantsmanship workshops.

Evidence: In accordance with statutes (c.f., Agricultural Research, Extension, & Education Reform Act of 1998) or other legislation & official policies, proposals are subject to rigorous peer and merit review including evaluation of the relevance or relative importance of subject matter categories of proposals. The agency maintains an extensive data base of highly qualified grant proposal reviewers from government, academia, & industry, & provides strict guidelines, training, & NPL & administrative oversight to the proposal review & award recommendation processes. All reviews are strictly confidential and are held to the same strict criteria for relevance, quality & performance as per the PMA. CSREES has a new Conflict of Interest and confidentiality form that has been approved by OMB (see Form 2007 - http://www.csrees.usda.gov/funding/forms_standard.html).This form and associated procedures should serve to avoid potential breaches of confidentiality and eliminate conflicts of interest.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The Agency comprehensively oversees grantee activities throughout the grant life-cycle. All competitive grant applicants are required to submit proposals fully describing the intended edu, research & extension work. This application is peer-reviewed for scientific merit and budgetary feasibility. If the application is approved a grant is awarded. If necessary, CSREES can place additional controls on new grant recipients. The activity's information is then loaded into the Current Research Information System (CRIS) (www.cris@csrees.usda.gov). Grantees must submit annual Progress/Accomplishments Reports for each funded activity, and a Termination Report upon its completion. NPLs provide program leadership & oversight of approved proposals through site visits, meetings with P.I.'s, P.I. seminars at CSREES, and participation in board/Council meetings of regional programs. Expert peer review panels are used periodically to assess grantees' efforts and link them to ongoing portfolio reviews. In addition, the agency tracks grant expenditures.

Evidence: Oversight practices & techniques provide CSREES with explicit knowledge of grantee activities. The Admin. Manual & agency regs contains explicit monitoring & reporting requirements. Grantee institutions submit required annual Progress Reports according to Award Terms & Conditions. Termination Reports must be submitted for each completed or terminated project. Progress reports on active projects incl. a summary of outputs & accomplishments including presentations & publications. Productivity on previous grants is an important criteria considered by proposal review panels. Since the nutrition sections of the NRI have a success rate of only 15-20%, only the best get funded. If researchers have not been productive in the past, it is highly unlikely they will receive another grant.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: CSREES annually collects, compiles and disseminates grantee performance data on grants in the form of annual reports and makes them publicly available in a transparent manner via the CRIS Web Site at www.cris.csrees.usda.gov. Grantees are contractually required to submit Annual Reports of Accomplishments and Termination Reports to describe their performance in achieving their stated goals including reference for presentations, publications, and websites. These reports are also selectively included in the NRI Annual Report and in feature stories from the Communications unit of CSREES which can be picked up by the media.

Evidence: The widely used CRIS is a comprehensive repository of performance data for all types of funded projects from CSREES including competitively awarded, formula funded & congressionally mandated projects. CRIS also includes the work of related agencies with similar missions & activities. These are identified by knowledge area, subject of investigation, & field of science, & include proposal summaries, annual performance reports, outcomes & impacts, termination reports, & publications. This information is searchable, available to the public, & has the capacity to produce various types of reports according to customer needs. See www.csrees.usda.gov/business/other_links/egov/csrees/cris.html. Another CSREES innovation, the Competitive Research, Education, & Extension Mgmt System (C-REEMS), makes comprehensive information on proposals, their review & award management available to NPLs to use in creating specialized reports. See www.csrees.usda.gov/business/other_links/egov/csrees/creems.html

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: Although, by law some proposals are allocated through other than competitive awards, that is through earmarks and formula grants, the internal review of the grantee proposals are consistent with the standards used for competitive awards. To ensure quality for other than competitive grants programs, such as formula funded, individual investigator & Multistate projects, extension programs & special grants, partners are required to submit annual plans of work & proposals subject to merit/peer review. At least 1 NPL & under certain circumstances 2 outside reviewers review & approve the proposal prior to award. The same rigor used to review proposals for competitive programs is used to ensure relevance, quality & performance for all CSREES programs. CSREES ensures that all proposals submitted include all required admin. & programmatic elements. Funds are withheld until all NPLs concerns are met before the project is approved. Each year CSREES publishes guidelines for NPLs on how to request proposals, review & approve projects for funding. Expert review panels have found no differences between competitive & other funded research, & have indicated all projects contribute to goals of CSREES, regardless of funding mechanism. CSREES funding allocation, review processes, & results are reviewed by the National Agricultural Research, Extension, Education, and Economics Advisory Board.

Evidence: Management of these programs is explicitly conducted per OMB Circulars for Federal assistance, USDA regs. & Administrative Manuals, CRIS Instructions, & C-REEMS instructions. Agency records indicate receipt & approval of financial & technical reports. Quantitative & qualitative assessments of outcomes of formula & special grants as well as competitive grants by panels during portfolio reviews provide focus & guide future efforts. Research projects supported with competitive& formula funds are reported in CRIS (See Admin. Manuals for Hatch, Evans-Allen, McIntire-Stennis). Institutions are required to conduct peer reviews for Special Research Grants. Grantees are required to conduct merit reviews for both the research & extension grants. See CSREES' 'National Program Leader Guidelines for Reviewing Hatch, McIntire-Stennis, Evans-Allen, & Animal Health & Disease Proposals', CSREES Customer Service Satisfaction Standards, 'Operational Guide for Trade Adjustment Assistance for Farmers'.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Based on the measures listed, the Nutrition Portfolio is demonstrating progress in achieving its three long term goals. The first goal is to raise the overall Nutrition Portfolio score to 91 by 2011. The external Portfolio Review panel gave it a high score of 86 in 2006. Although improving this will be difficult, the panel had praise for the staff and the direction of future work. The second goal is to continue to see 93% of EFNEP graduates improve their diets. Although EFNEP success is dependant on the collective efforts of programs across the country, past success indicates future success. The third goal is to increase the development and use of new interventions to improve diets and physical activity by 1 per year until 2010. The highly successful EFNEP intervention model which has evolved based on research and experience is cited as the baseline for the third goal. Guided by panel recommendations & stakeholder input, obesity prevention interventions will be the focus in the future.

Evidence: See measures tab.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The Nutrition Portfolio has 3 specific annual performance measures which have been met & which demonstrate progress in meeting LT goals. For the 1st measure, self-assessment of the portfolio, the staff gave the portfolio a score of 80 in February 2006 - 6 points below the score given by the external panel. Data for the 2 other long term measures - % of EFNEP graduates improving their diet & the development & use of nutrition interventions - are collected annually to ensure program progress. Indications from the external panel review & oversight by the Deputy Administrator are that work is on track & goals will be achieved. The 2 other annual measures are for efficiency in reviewing proposals. So far, for 2006, targets for these have been met.

Evidence: It is clear from the independent external expert panel review's comments and scores that the portfolios, its programs, and partners are likely to achieve their annual performance goals. The efficiency target for time per proposal processed, 216 days, was met in 2005 and looks like it will meet its target of 212 days in 2006. The other efficiency target met its 2006 target of $535 per proposal reviewed, in 2005, a year early.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Both efficiency targets were met in FY2005. One measure improved so much that it met its later FY target as well. The efficiency of the Agency's programs & processes are expected to continue to improve by reducing time-to-completion of proposal solicitation, review & funding, & minimizing peer vetting costs per proposal reviewed. As well as the transition to the full electronic solicitation, submission, review, & processing of proposals that will continue into 2006. This major change in the Agency's business process is anticipated to yield considerable efficiency & cost-effectiveness. The 2006 External Expert Panel Review report & score for the Portfolios also indicated considerable progress in outcomes. Factors that are playing a role in this are larger and longer term awards which reduce the relative cost of processing, days, and cost to grantees per award. A 25% increase in honoraria was to remain competitive with other research agencies in attracting qualified peer reviewers, & increased use of more effective but more costly on-site reviewers. In addition, both efficiency measures met their targets in 2005.

Evidence: Estimates at the end of 3rd quarter FY05 show the Agency has made improvements in reduction in time from submission to receipt of grant award & associated costs. Efficiency benchmarks were reconfigured in '04 to reflect significant changes: increased air transportation costs, a 25% inc in honoraria to remain competitive with other research agencies in attracting qualified peer reviewers, & increased use of more effective but more costly on-site reviewers. Even with these adjustments, CSREES cost per proposal reviewed is extremely efficient vs. comparable NSF costs of $650 per proposal. The continuing trend toward focused, larger grant awards increases admin efficiency. CSREES honoraria are low compared to other federal agencies whom CSREES competes for highly qualified reviewers. The Nutrition Portfolios have made efficient & effective improvements & supporting programs have too. EFNEP has been found to save $10.64 in health care costs & $2.48 in food expenditures for every $1 spent.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: CSREES with other USDA agencies (ARS & FNS), HHS & other public & private entities contribute to the goal of improving the nutrition & health of Americans. However, CSREES is unique because it works in partnership with the university system to ensure that the most qualified university researchers work on the most important Administration priorities, that Ext uses the most effective interventions to reach the greatest number of vulnerable people & that the next generation of researchers & practitioners has the ed & experience they need. The expert panel that reviewed the Nutrition Portfolio gave it an overall score of 86 - high compared to other agency portfolios. The EFNEP evaluation system provides on-demand reports to states, allowing them to closely monitor progress & improve outcomes.

Evidence: The 8 external panelists for the Nutrition Portfolio review were selected because they are experts in the field & very familiar with other Federal, state, university & private sector entities with a similar goal of improving the Nation's nutrition & health. The panel reviewed the portfolio on the basis of R&D criteria - relevance, quality & performance - & gave it a score of 86 - high compared to other agency portfolios (http://www.csrees.usda.gov/business/reporting/portfolio.html). This attests to the strength of the CSREES program. EFNEP is a very effective education intervention carried out in communities across the country, e.g., an independent study by the Produce for Better Health Foundation cited EFNEP as the most effective Federal program for increasing fruit & vegetable intake.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Nutrition Portfolios were independently assessed by outside experts for relevance, quality & performance &to assess their effectiveness in improving nutrition & health of Americans. Research affects productivity & public health with a time lag; benefit streams typically extend beyond 30 yrs. Studies found rates of return of 40-60% for public R&D regardless of level of aggregation or geographical area. Extension reduces time lags; its major effect on productivity is in the short to intermediate period, although major technology adoption throughout the sector are a long-term accomplishment. The # of citations reflects high quality research, as scientists most often cite seminal work in a field. CSREES funded research is used by others to replicate programs & disseminate information. Findings are used in Congressional testimony, & with industry professionals in decision making, program admin & governance.

Evidence: EFNEP has been evaluated for effectiveness. Studies using EFNEP's interventions have found the technique effective in improving eating & fitness choices & have been adopted in edu efforts. EFNEP has also been tested using various theoretical models to improve reach & impact of nutrition edu for low income Hispanic participants receiving the La Cocina Saludable curriculum. Iowa developed a standardized "Guide to Evaluating Written Nutrition Edu Materials". A pictorial set of food photos was developed as an aid in collecting accurate food recall data. California used pictorial intervention to work with low literacy audiences, & improved participant comprehension. GAO selected EFNEP as having effective program evaluation GAO-02-923, 2002; Produce for Better Health Foundation identifies EFNEP as the most effective program in increasing fruit & vegetable intakes - The Fruit & Vegetable Consumption Challenge: How Federal Spending Falls Short of Addressing Public Health Needs, 2002. EFNEP has been evaluated for effectiveness. Studies using EFNEP's interventions have found the technique effective in improving eating & fitness choices & have been adopted in edu efforts. EFNEP has also been tested using various theoretical models to improve reach & impact of nutrition edu for low income Hispanic participants receiving the La Cocina Saludable curriculum. Iowa developed a standardized "Guide to Evaluating Written Nutrition Edu Materials". A pictorial set of food photos was developed as an aid in collecting accurate food recall data. California used pictorial intervention to work with low literacy audiences, & improved participant comprehension. GAO selected EFNEP as having effective program evaluation GAO-02-923, 2002; Produce for Better Health Foundation identifies EFNEP as the most effective program in increasing fruit & vegetable intakes - The Fruit & Vegetable Consumption Challenge: How Federal Spending Falls Short of Addressing Public Health Needs, 2002.