Detailed Information on the
Food Stamp Nutrition Education Assessment

Develop and finalize Education and Administrative Reporting System (EARS).

Identify and develop efficiency performance measures based on preliminary findings from EARS.

Draft a plan to promulgate regulations designed to increase State accountability for program administration and evaluation.

Implement EARS across all FSNE programs; monitor State performance.

Begin EARS rollout.

Seek legislation to make nutrition education a component of the Food Stamp Program

Draft a plan to increase the use of evidence-based food and nutrition education programs across States.

Measure: Total FNSE Participation

Explanation:The Education and Administrative Reporting System (EARS) is a data collection tool that will report FSNE participation. Implementation is scheduled for FY2008. Baseline will be established in December 2009, with annual reporting thereafter.

Measure: FSNE Participation Rate in Direct Education Programs among School-Age Children

Explanation:FSNE participation rate among school-aged children is a proxy for how well the program targets children, who represent nearly half of all FSP participants. The numerator is the number of school-aged FSNE participants. The denominator is the total number of FSNE participants. Baseline data will be collected upon implementation of EARS and reported annually thereafter.

Measure: FNSE Participation Rate among Women

Explanation:FSNE participation rate among women is a proxy for how well the program targets women, who tend to do more of the food purchasing for FSP households. The numerator is the number of female FSNE participants. The denominator is the total number of FSNE participants. Baseline data will be collected upon implementation of EARS and reported annually thereafter.

Measure: Timely implementation of EARS

Explanation:New performance measures will be developed based on data gathered using EARS. Therefore, timely implementation is a required for FSNE to meet its performance monitoring goal.

Measure: Ratio of FSNE administrative costs to FSNE delivery costs

Explanation:The numerator is expenditures related to administering FSNE (i.e., costs related to overhead, office supplies, training providers on administrative tasks). The denominator is expenditures related to delivering FSNE (i.e., costs related to purchasing and/or developing educational materials, evaluation, training providers on FSNE activities)

Measure: Ratio of FSNE Direct Education Participants Receiving Food Stamps to All FSNE Direct Education Participants

Explanation:The numerator is the total number of FSP recipients who participate in FSNE direct education. The denominator is the total number of FSNE participants in direct education. This efficiency measure is calculated based on data collected by the Educations Administrative Reporting System (EARS) form (Form Approved OMB No. 0584-0542), which is being implemented by States over a two-year period starting in FY2008. By FY2010, all States will be collecting applicable EARS data with a submission deadline of 12/31/2010. During this two-year implementation phase, State agencies and implementing partners are expected to create or modify elements of their current data collection systems to that by the end of the implementation period, reliable data can be submitted on all applicable elements of the EARS form. The agency is in the process of developing reliable baseline data for this measure, which is contingent on implementation of EARS. The baseline for this efficiency measure will be available once implementation of EARS is complete and EARS data is fully scrutinized.

Measure: Ratio of Persons Reached by FSNE Social Marketing Who Receive Food Stamps to All Persons Reached by FSNE Social Marketing

Explanation:The numerator is the estimated total number of FSP recipients who are reached by FSNE social marketing. The denominator is estimated total number of persons reached by FSNE social marketing. This efficiency measure is calculated based on data collected by the Educations Administrative Reporting System (EARS) form (Form Approved OMB No. 0584-0542), which is being implemented by States over a two-year period starting in FY2008. By FY2010, all States will be collecting applicable EARS data with a submission deadline of 12/31/2010. During this two-year implementation phase, State agencies and implementing partners are expected to create or modify elements of their current data collection systems to that by the end of the implementation period, reliable data can be submitted on all applicable elements of the EARS form. The agency is in the process of developing reliable baseline data for this measure, which is contingent on implementation of EARS. The baseline for this efficiency measure will be available once implementation of EARS is complete and EARS data is fully scrutinized.

Is the program purpose clear?

Explanation: The goal of Food Stamp Nutrition Education (FSNE) is clearly defined in the Food Stamp Nutrition Education Plan Guidance: "To improve the likelihood that persons eligible for the Food Stamp Program (FSP) will make healthy food choices within a limited budget and choose physically active lifestyles consistent with the current Dietary Guidelines for Americans (DGAs) and My Pyramid" (Page 1 of Guidance). Section 2 of the Food Stamp Act of 1977 and Program regulations at 7 CFR 271.1 set forth the purpose of the FSP. As stated in the regulations, the purpose of the program is: "To promote the general welfare and to safeguard the health and well-being of the Nation's population by raising the levels of nutrition among low-income households." As such, the legislative goals are directed to the FSP as a whole, rather than to its many subcomponents, including nutrition education. Thus, while nutrition education is clearly intended to contribute to advancing the program's purpose, the legislation and regulations are silent on the specific goals of nutrition education. Although nutrition education within FSP is small in comparison to the overall program investment, FSNE still represents a significant investment of Federal funds. As a part of the current legislative recommendations for the 2007 Farm Bill, FNS proposes to amend existing legislative language governing the nutrition education aspect of the program to make the goals and objectives of nutrition education stronger and more prominent in the Food Stamp Act, while removing obsolete references to past operations, such as early grant opportunities. USDA's strategic performance plan calls for increasing HEI scores by about 5 percent for Americans with incomes less than 130 percent of poverty, essentially those who meet the FSP's gross income limit. HEI is a measure of diet quality that assesses conformance to the DGAs. The FSP is the major component in USDA's efforts to help low-income Americans reach that goal. Food stamp benefits provide families with the resources to purchase a healthful diet and Food Stamp Nutrition Education (FSNE) is the Program tool to encourage food choices that are consistent with the latest DGAs. (Guidance Introduction- page1 and 5).

Evidence: Food Stamp Act http://www.fns.usda.gov/fsp/rules/Legislation/pdfs/197fsa.pdf FSP regulations (7CFR 271.1); (7CFR 272.2(d));Electronic Code of Federal Regulations: Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008. USDA Strategic Plan for 2005-2010. Food Stamp Nutrition Connection: National FSNE -- Food Stamp Education Plan Guidance and Templates

Does the program address a specific and existing problem, interest, or need?

Explanation: Approximately two-thirds of adults in America and nearly one-fifth of children and adolescents are overweight??putting them at risk for serious health problems including heart disease, hypertension, diabetes and some cancers. Historically, low-income households have a higher prevalence of health conditions related to poor nutrition than households with higher incomes. While recent data suggests the gap between low and higher income women may be narrowing, this effect is due to an increase in overweight and obesity in higher income women, and rates of obesity among low-income women remain high. While obesity rates have doubled in children and tripled in adolescents over the last two decades, these rates have increased the most among those in the lowest income levels, especially African American and Mexican American children. Research on nutritional intake of persons with incomes less than 130 percent of poverty indicates that HEI scores average about 62 out of a total of 100. This score is at the lower end of the "Needs Improvement" range for the HEI. Further, research also indicates that many low-income adults do not know specific facts related to healthful dietary practices, such as the types of foods they should eat to maintain a healthy diet. Less than one-third of adults engage in regular leisure-time physical activity, and lower income adults are less likely to do so than higher-income adults.

Evidence: Gleason P, Rangarajan A, Olson C. Dietary Intake and Dietary Attitudes Among Food Stamp Participants and Other Low-Income Individuals. USDA, September 2000. National Center for Health Statistics, Health, United States, 2006 with Chartbook on Trends in the Health of Americans. Hyattsville, MD: 2006 http://www.cdc.gov/nchs/data/hus/hus06.pdf Berkowitz RI, Stallings VA, Maislin G, Stunkard AJ. Growth of children at high risk for obesity during the first 6 y of life: implication for prevention. Am J Clin Nutr. 2005; 81:140-6. The Surgeon General's Call To Action To Prevent and Decrease Overweight and Obesity 2001, U. S. Department of Health and Human Services, Public Health Service, Rockville, MD. Ogden CL, Flegal KM, Carroll MD, Johnson CL. Prevalence and trends in overweight among US children and adolescents, 1999-2000. JAMA 288: 1728-32. 2002. Health, United States, 2002, National Center for Health Statistics, CDC

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: FSNE is designed to reinforce and coordinate with other Federal state and local efforts targeting nutrition education to low-income families. To improve the food choices of the target population, it is essential to deliver reinforcing messages through multiple channels of communication. Nutrition education initiatives delivered through FSNE, the Special Supplemental Nutrition Program for Women Infants and Children (WIC) program, the Expanded Food and Nutrition Education Program (EFNEP), and school meal programs each target different, but overlapping, segments of the low-income population. Education, outreach and training efforts help the target population acquire the knowledge, skills, attitudes, and behavior necessary for nutritionally sound diets. This is consistent with basic behavior change theory, which emphasizes the importance of delivering consistent messages from multiple sources. FSNE targets those persons who participate in or who are eligible to participate in FSP, while EFNEP is designed to educate low-income families and youth. EFNEP is typically delivered in small groups or one-on-one counseling, while FSNE is often delivered through a broader range of communication approaches, including social marketing initiatives. There is a large gap between the number of people served by the EFNEP (563,000 participants in FY2005) and the size of the low-income population. By comparison, USDA estimates that there were 38 million persons eligible for food stamps in 2004. Further, USDA has set forth appropriate safeguards for avoiding unnecessary duplication to ensure that only additional coverage of activities funded under EFNEP may be claimed under a FSNE grant. Such activities must expand a State's current EFNEP coverage and serve additional FSP eligibles or provide additional educational services to EFNEP clients who are eligible for the FSP. USDA's Cooperative State Research, Education, and Extension Service (CSREES), the primary contractor providing FSNE within States, uses the EFNEP model in delivering services to FSNE participants. FSNE Guidance requires that providers consider existing educational materials before purchasing or developing new ones. FNS created a web site, the Food Stamp Nutrition Connection, which serves as a library for States to place curricula, materials and other FSNE-developed resources. States may use the site to access other States' materials and activities to capitalize on existing resources. (Guidance; C.4, Allowable costs page 68) In general, Food Stamp State agencies must ensure that FSNE activities avoid duplication of existing services by conducting a needs assessment of the State FSNE target audience. The State plan must reflect the results of this needs assessment, which also provides States with a means of identifying potential partners for collaboration and areas of the State where the target audience is underserved. State agencies are also encouraged to promote coordination and collaboration among a variety of stakeholders at the local, State, regional and national levels, and to identify specific roles and responsibilities for local, State, regional and national food stamp agencies and nutrition education providers. FNS encourages the use of State Nutrition Action Plans (SNAP) to connect nutrition education across programs and to facilitate a collaborative and integrated nutrition education approach. SNAPs identify a common nutrition goal within a State and contain objectives, strategies and tactics for achieving that goal in a unified manner. Further, FSNE guidance specifically prohibits use of FSNE funds for activities funded under other Federal programs. (Guidance; C.4 page, Unallowable costs, page 70.)

Evidence: Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008, FNSE Guiding Principles. SNAP process: http://www.fns.usda.gov/oane/SNAP/SNAP.htm.) EFNEP Description/Data: Expanded Food and Nutrition Education Program Food Stamp Eligibles Estimate: http://www.fns.usda.gov/oane/MENU/Published/FSP/FILES/Participation/FSPPart2004-Summary.pdf Other major federal nutrition education initiatives, including MyPyramid.gov, MiPiramide, and MyPyramid for Kids web sites, and the DGAs, as well as the food labeling policies of the Food and Drug Administration provide information to the general public and are not specifically designed for low-income audiences. FSNE supports and leverages these efforts by incorporating the guidance in materials and activities that are relevant and accessible to low-income populations considering both literacy and culturally-relevant messages.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no empirical evidence suggesting the program design is flawed or effective. Nor is there strong evidence showing that another approach (i.e. WIC model) would be more efficient or effective. The current statutory language governing nutrition education is broad and open-ended. FNS relies on the program's recently-published Guiding Principles to encourage, but not require, education features that have been shown by the research literature to be effective. These include behaviorally focused messages, use of motivators and reinforcements that are personally relevant to the target audience, use of multiple channels of communication to convey messages, approaches that provide for personal engagement in learning activities, and intensity/duration that provides for multiple exposures to the same message. State FSP agencies build upon existing nutrition education resources within each state by contracting with Cooperative State Research, Education and Extension Service (CSREES), State Departments of Health or Education, and State-level nutrition networks or others to provide nutrition education services to the target audience. (Guidance, Appendix H, pages 85-93) Federal FSNE funds are provided as a 50% reimbursement of allowable state expenditures. Thus, States have a strong incentive to ensure that funds are spent efficiently and effectively.

Evidence: Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008, FNSE Guiding Principles Nutrition Education Literature: Isabel Contento, G. Balch, Y.L. Bronner, L.A. Lytle, S.K. Maloney, S.L. White, C.M. Olson and S.S. Swadener The Effectiveness of Nutrition Educationand Implications for Nutrition Education Policy, Programs, and Research: A Review of Research. Journal of Nutrition Education, 27(6), 277-422, 1995

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The Food Stamp Act, as amended, indicates that FSNE activities should be directed to persons eligible for the FSP. FNS has prioritized categories of potential recipients of State FSNE activities. The three categories of FSNE recipients are: certified eligibles, likely eligibles, and potentially eligible by site/location. States must request a waiver for FSNE projects which are targeted at the latter category, and this audience may receive FSNE only when it is not possible or practical to distinguish Program eligibles and/or identify Program eligibility. (Guidance Pages 8-11). All FSNE activities, including those in waiver sites, are targeted at low-income persons who are FSP-eligible. At least half of the persons who receive FSNE in such projects must have gross incomes at or below 185% of the poverty guidelines. A review of FSNE activities for FY2004 found that 74% of FSNE participants were FSP participants or income-eligible non-participants. Further, as a matter of practice, State and local FSNE programs tailor their efforts to address specific State and local needs as identified by a thorough needs assessment. State and local projects reflect specific goals in terms of improving diet and increasing compliance with the DGAs (and therefore HEI scores). (Guidance, Describing the findings of your needs assessment; page 12)

Evidence: Food Stamp Act, 7 USC 2011, Section 11 (f) (1)).Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008. Abt Associates, Food Stamp Nutrition Education Systems Review Final Report. USDA, September 2006. Food Stamp Nutrition Education State Plans.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program is adopting three new participation measures, which reflect both short-and long-term goals. Due to data limitations, the program is unable to use directly-related outcomes to measure performance. The first output performance measures is FSNE participation. Since the purpose of the program is to encourage the purchase, preparation, and consumption of nutritious foods, changes in FSNE participation will reflect the program's overall ability to engage low-income individuals in their education activities to change behavior. The second and third measures will assess efforts to target FSNE to selected populations, namely school-aged children and women. Nearly half of all FSP participants are children, and the majority of purchases among FSP households are made by women. Since the purpose of FSNE is to provide education to FSP and FSP eligible individuals, these measures will help determine whether nutrition education efforts reach the intended audience. Upon implementation of EARS, the program will be able to collect these performance measures, as well as develop outcome-based performance measures. EARS has been under development for several years with implementation expected to begin in FY2008 and full data collection to start in FY2010. The exact nature of the performance measure will be determined when the system is finalized.

Evidence: USDA Strategic Plan for 2005-2010. Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008. HEI: http://www.cnpp.usda.gov/Publications/HEI/healthyeatingindex2005factsheet.pdf Education and Administrative Reporting System OMB Information Collection Approval Submission

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Baseline data for the three performance measures are not established. State and local FSNE providers identify their own sub-goals and performance measures that are related to improving conformance with the DGAs, but focused on the specific needs and characteristics of their individual target populations. While some of the objectives specified in provider plans are more ambitious than others, taken together, the goal and objective setting at each level contribute to meeting the overall ambitious aim of enabling low-income families to eat a manner that is consistent with the Dietary Guidelines. USDA does not yet have targets and timeframes for a FSNE participation measure. Goals and timeframes will be established once baseline data is available, which will occur after the implementation of EARS.

Evidence: USDA Strategic Plan for 2005-2010; FSNE State Plans. Education and Administrative Reporting System OMB Information Collection Approval Submission

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program is adopting three new participation measures, which reflect both short-and long-term goals. Due to data limitations, the program is unable to use directly-related outcomes to measure performance. The first output performance measures is overall FSNE participation. Since the purpose of the program is to encourage the purchase, preparation, and consumption of nutritious foods, changes in FSNE participation will reflect the program's overall ability to engage low-income individuals in their education activities to change behavior. The second and third measures will assess efforts to target FSNE to selected populations, namely school-aged children and women. Nearly half of all FSP participants are children, and the majority of purchases among FSP households are made by women. Since the purpose of FSNE is to provide education to FSP and FSP eligible individuals, these measures will help determine whether nutrition education efforts reach the intended audience. Upon implementation of EARS, the program will be able to collect these performance measures, as well as develop outcome-based performance measures. EARS has been under development for several years with implementation expected to begin in FY2008 and full data collection to start in FY2010. The exact nature of the performance measure will be determined when the system is finalized. Additionally, the program has developed a short-term performance measure that will assess timely implementation of EARS. This is an important, short-term output measure because all performance goals are based on EARS, which will provide annual uniform data from States on FSNE activities. EARS is pending OMB approval, and data collection is expected to begin in 2008. State and local FSNE providers also identify their own annual performance measures related to improving conformance with the DGAs among the specific groups they are targeting. Each provider component of FSNE monitors their own progress on their specific measures, and the combined effort contributes to the achievement of the overall program goal. (Guidance, Section B, Page11, Appendix A, Template 2)

Evidence: Education and Administrative Reporting System OMB Information Collection Approval Submission; Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: USDA does not yet have targets and timeframes for its FSNE participation measures. Goals and timeframes will be established once baseline data is available, which will occur after the implementation of EARS. While, the FY 2004 assessment provided data to show that an estimated 67 percent of persons receiving food stamp nutrition education were school-aged children (42 percent) or adult women (25 percent), this was voluntary data and it is not clear that it could serve as a baseline. EARS is expected to provide annual uniform data from States on the targeting of FSNE services to these groups. FNS has a detailed multi-year implementation plan for EARS and is currently on schedule for initial data collection to begin 2008 and full data collection to start in FY2010. The information collection approval submission is currently pending OMB approval.

Evidence: Bell, L., Tao, F., et al. (2006) Food Stamp Nutrition Education Systems Review: Final Report. U.S. Department of Agriculture, Food and Nutrition Service. EARS OMB Information Collection Approval Submission

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: FSNE operations at all levels work to achieve specific goals which contribute to increased conformity with the DGAs, and thus contribute to overall USDA goals to improve diet quality. FSNE planning and delivery is coordinated at the Federal, State and local levels in order to work toward annual and long-term goals. Although FSNE is an option in the FSP, 52 State agencies conduct FSNE activities in FY2007, an increase from 7 in 1992. An assessment of FSNE activity in FY2004 found significant collaboration between State implementing agencies and local projects in planning and decision-making. The Food Stamp Nutrition Education Guiding Principles are part of the annual guidance FNS provides to States for preparation of their FSNE State Plans. The FSNE State Plan Guidance establishes clear standards for achieving the program goal of improving the likelihood that persons eligible for Food Stamps will make healthy food choices within a limited budget and choose physically active lifestyles consistent with the current DGAs and My Pyramid. States then work with their contracted FSNE providers to set goals and objectives consistent with the guidance. (Guidance Appendix H pages 85-93) State Plans of Operation for each fiscal year are due by August 15 of the year prior, and are subject to FNS approval for reimbursement. State plans are reviewed for consistency with the Guidance. In addition, an Annual Final Report of each State's nutrition education activities is required by the FSNE Guidance (Guidance pages 6-8). The Guidance requires States to summarize the nutrition education projects implemented in the previous fiscal year, including outcomes and effectiveness of the FSNE program and how the program might be improved in the upcoming FY. The reports include geographic area served, number and type of delivery sites, target audience, total number of participants, frequency and duration of education sessions, the key messages taught and what types of evaluations were completed for each project. For social marketing, the final reports provide the name of the campaign(s), the duration and target audience of the campaign, the primary activities and key messages used as well as the types of evaluations completed for each campaign. The Final Reports are due to FNS by November 30th of every year for the previous FY. By May 2008, FSNE will develop and submit a plan to OMB as to how the Annual Report will be integrated with EARS data collection and reporting processes.

Evidence: Bell, L., Tao, F., et al. (2006) Food Stamp Nutrition Education Systems Review: Final Report. U.S. Department of Agriculture, Food and Nutrition Service; U.S. Department of Agriculture, Food and Nutrition Service (2005) Food Stamp Nutrition Education Guiding Principles. U.S. Department of Agriculture, Food and Nutrition Service.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The agency does not conduct non-biased evaluations of the program on a regular or as-needed basis. In 1997 and 2004, the agency conducted a national review of FSNE operations and decision-making activities; however, these reviews are not sufficiently rigorous to provide information on the effectiveness of the program. While FSNE providers are encouraged to conduct evaluations of their specific activities to inform program planning and improvement, most providers complete process evaluations that do not show the impact or desired outcomes of their activities. Most providers track implementation through collection of participation or project data. While no high quality, independent outcome evaluations have been conducted at the national level, some evaluations have been conducted on interventions which reflect large proportions of FSNE activities. For example the State of California (which accounts for approximately 40% of FSNE expenditures) conducts statewide surveys providing detailed information about California children's fruit and vegetable consumption and physical activity and has used this data to assess the impacts of its FSNE activities throughout the state. These evaluations are based on data from the California Children's Healthy Eating and Exercise Practices Survey, a statewide surveillance project of the California Department of Health Services conducted biennially. In addition, the USDA Cooperative State Research, Education, and Extension Service (CSREES), the primary contractor providing FSNE within States, sponsored an assessment of their national FSNE activities in FY 2002. However, this was an assessment of self-reported data and performed by a participant, the Mississippi State University Extension Service. CSREES also collects nutritional behavioral change data from EFNEP participants on an ongoing basis. CSREES' system for ongoing monitoring and evaluating the impacts of EFNEP was recognized in the 2006 PART review for its Grants for Nutrition and Health, and is the source of the long-term performance measure for EFNEP. The PART review also noted that "because EFNEP is the model for much of the larger FSNE program, it is indicative of the success of that program as well." EFNEP is used as a model by Cooperative Extension Service FSNE providers, who comprised 55% of FSNE providers in FY2004.

Evidence: Anliker, J., Bell, L., et al. (2000) Food Stamp Nutrition Education Study: Final Report. U.S. Department of Agriculture, Food and Nutrition Service; Bell, L., Tao, F., et al. (2006) Food Stamp Nutrition Education Systems Review: Final Report. U.S. Department of Agriculture, Food and Nutrition Service; Little, D. and Newman, M. (2003) Food Stamp Nutrition Education within the Cooperative Extension/Land-Grant University System: National Report FY 2002. U.S. Department of Agriculture, Cooperative State Research, Education, and Extension Service. http://www.csrees.usda.gov/nea/food/fsne/pdfs/fsne_national_report_2002.pdf Presentation by California Department of Health Services, Association of State Nutrition Network Administrators, Winter 2007 meeting. CalCHEEPS description: http://www.dhs.ca.gov/ps/cdic/cpns/research/download/calcheeps_2003/calcheeps_Background_Doc.pdf CSREES 2006 PART review: https://max.omb.gov/app/part/program/assessment?pid=2476&aid=4426

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: FSNE does not have program budgeting in place that defines the relationship between 1) annual and long-term performance targets and 2) resources. In addition, FSNE does not have evidence to show that the requested performance/resource mix will enable the program to achieve its performance goals. Finally, the program does not report all direct and indirect costs needed to meet performance targets.

Evidence: The Food Stamps Program PART also received a No for this question with this explanation, "Requests for additional Federal funding for payment accuracty and nutrition education have failed to tie the level of resources requested with measurable improvements in program performance."

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: FSNE recognized the need to better focus FSNE efforts and in 2004-05 engaged in a broad based policy development effort with all stakeholders and interested parties. In September 2005, FNS released the Food Stamp Nutrition Education Guiding Principles, a policy document which identifies standards of excellence in food stamp nutrition education that have been integrated into FSNE Guidance for the States. FNS also requires an annual on-site review of State FSNE projects based on one or more of a list of criteria. (Guidance page 4) The objective of the FSNE Management Evaluation is to ensure both fiscal and program integrity. Data gathered through this process provides a basis for assessing the administration of FNS program operations, planning future management objectives, and identifying any needed corrective action. In addition, FNS is implementing an ongoing reporting system for FSNE, which will provide uniform data and information about the nutrition education activities of all participating States. EARS will improve FNS' ability to assess annual performance measures relative to annual performance goals by establishing ongoing and consistent reporting of basic program administrative data. The proposed EARS data collection form is currently pending OMB information collection approval; initial data collection is to begin in FY2008. USDA is also actively engaged in pursuing improvements in evaluation methods and long-term performance measures available for FSNE activities. FNS has formed the USDA interagency Working Group on FSNE Evaluation, along with CSREES, Economic Research Services (ERS), and Agricultural Research Services (ARS). In 2003, two workshops were held focusing on FSNE research and evaluation. ERS has published a compendium of 128 survey questions used in previous research to assess dietary knowledge, attitudes, and behaviors for low-income populations. This report is part of a joint ERS-FNS research effort to develop a common core set of questions to assess the dietary behavior impact of FSNE on FSP participants. FNS and ERS also sponsored a series of papers to support evaluation of nutrition education provided through FSNE activities, published in the Journal of Nutrition Education in 2001. Finally, FNS developed a technical assistance tool for nutrition education evaluators that provides a comparison of different kinds of evaluation, principles of credible impact evaluations, and links to web sites with more detailed instruction. This information has been provided through presentations at a variety of meetings, and through electronic posting. It was also used to strengthen the evaluation guidance for FSNE State plans in FY2007 and 2008 in order to improve the quality of evaluation conducted at the State and local levels.

Evidence: EARS OMB Information Collection Approval Submission; Management Evaluation Reports. Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008. Prototype Notebook: Short Questions on Dietary Intake, Knowledge, Attitudes, and Behaviors By Heather Hartline-Grafton, Rebecca Nyman, Ronette Briefel, and Rhoda Cohen, Mathematica Policy Research, Inc. September 2004. Prototype Notebook: Short Questions on Dietary Intake, Knowledge, Attitudes, and Behaviors Priorities and Planning Workshop - Future FSNEP Research and Evaluation Activities, hosted by the USDA Economic Research Service, January 15-16, 2003 Advancing the Practice of Food Stamp Nutrition Education Evaluation: Art and Science, hosted by the Society for Nutrition Education, July 30-31, 2003. Nutrition Education: Principles of Impact Evaluation. Office of Analysis, Nutrition and Evaluation, FNS, USDA. September, 2005. http://www.fns.usda.gov/oane/MENU/Published/NutritionEducation/Files/EvaluationPrinciples.pdf

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: While the program collects final reports for nutrition education activities from FSNE partners annually, FSNE does not use the information to manage the program and improve grantee performance. States use the final reports to summarize nutrition education activities and impact evaluations. However, data reported is not consistent and states use varying definitions. This is the main reason the program developed EARS. The program is legislatively mandated to monitor program administration and operations; however, it is not legislatively mandated to manage program performance. The program does not use information from final reports to adjust program priorities or allocate resources.

Evidence: Food Stamp Nutrition Education Plan Guidance, Management Evaluation Review of State FSNE Projects, Page 4, Federal Fiscal Year 2008. (see Appendix A. Template 1. Section A and B of 2008 Guidance). (see Appendix A. Template 2. Part 2. Section i. 2008 Guidance). EARS OMB Information Collection Approval Submission

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: All Federal managers and Program partners are held strictly accountable. As part of the State Plan process, State agencies submit to FNS for approval a Budget Projection Statement (BPS) that provides projection of the total costs for major areas of program operations. The BPS is submitted annually by August 15th, for the upcoming fiscal year and is updated as necessary throughout the year and any time additional Federal funding is needed. Officials of the State food stamp agency must sign the FSNE plan attesting to the accuracy and financial efficacy of the plan and submit it to FNS for review and approval annually no later than August 15 for the following fiscal year's operation. One condition of approval is that a plan must have measurable objectives and the activities proposed must support those objectives. States are held accountable for the content of their nutrition education plan and for the subsequent implementation of the plan. Once approved, State cooperators provide annual end-of- year reports to FNS on their educational strategies and efforts undertaken over the past year. Federal managers are responsible for ensuring that management evaluations to assess and promote compliance with FSNE standards are conducted and that identified deficiencies are addressed. FNS managers are required to have their performance plans aligned with USDA strategic goals and FNS priorities. Agency work responsibilities related to annual and long-term program goals, emerging problems, and other factors are identified and planned prior to the beginning of the fiscal year. Performance plans explicitly tie managers' responsibilities to accomplishment of this designated priority work, including program oversight and conduct of management evaluations, and ensuring implementation of corrective action, along with the strategic and annual goals this work supports. The budgetary and accounting process is strictly controlled with specific dates for information to be submitted and verified. Specific guidance on allowable costs, indirect costs, and cost allocations has been established based on Government wide requirements. Additionally, FSP regulations at 7 CFR 277.11 require quarterly expenditure reporting via the SF-269 form. This form has a separate column for reporting nutrition education and includes both the State and Federal share of the costs, and both current expenditures and unliquidated obligations.

Evidence: FSP regulations at 7 CFR 272.2 (c), Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008. OMB Circulars A-21 (Universities and Colleges) and A-87 (State and Local Governments). Memorandum to Subcabinet and Agency Heads from USDA Deputy Secretary on President's Management Agenda and Performance Standards of GS-14s and 15s; Attestation of Under Secretary Bost to compliance with requirements; COMPASS planning system

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Funds are obligated by both Federal and State partners in a timely manner. State Plans of Operation for each fiscal year are due by August 15 of the year prior and are subject to FNS approval for reimbursement. When a FSNE State plan is approved, the State submits an FNS-366A Budget Projection to project funding needs. When the budget is approved, FNS notifies the State in writing of the approved amount of Federal funding. Over the past three years the percentage of plans approved by October 1 has improved substantially and consistently -- FY 2005 52%; FY2006; 68% and FY 2007 94%. This improvement helps ensure that funding is obligated in a timely manner. Budget requests are normally approved by the start of the Federal Fiscal Year and quarterly expenditure reports are due one month after the close of the Federal Fiscal Quarter. Funding levels are reviewed for conformance with the plan budget and differences are referred back to the state for clarification. Significant in-depth analyses of the allowability of proposed operating costs are made during the plan review process. Both audits and management evaluations are conducted to insure that the planned and approved activities are conducted in accordance with the allowable cost guidance. Any costs that do not meet this guidance are disallowed, and such costs are borne fully by the State agency. For example, a review of one state for 2006 resulted in the disallowance of expenditures for foods that were purchased for nutrition education purposes, including items such as bacon, caramel popcorn balls, and mayonnaise and cheese whiz. It is the intention and priority of FSNE to put emphasis on healthier food choices. For that reason, these foods are inappropriate and should not be paid for with FSNE funds. States must complete two or more reviews of subgrantees each year.

Evidence: FSNE State Plans; FNS-366A Budget Projections, Management Evaluation Reports.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does not use cost-effectiveness data to drive its programmatic or funding decisions. While the agency recently developed guidance on improving FSNE State program operations, there is no data indicating State program effectiveness.

Evidence:

Does the program collaborate and coordinate effectively with related programs?

Explanation: An April 2004 GAO report on Nutrition Education by USDA suggested that stronger linkages among various USDA efforts including FSNE were needed. FNS has made a concerted effort to strengthen linkages among its primary FSNE partners, such as USDA Cooperative State Research Education and Extension Service (CSREES), National Agricultural Library Food and Nutrition Information Center(NAL-FNIC) and Association of State Nutrition Network Administration (ASNNA) by providing joint National trainings, participating as presenters in individual organizational conferences and participating in ongoing meetings with a Federal Interagency Working Group on FSNE Evaluation, to name a few primary efforts. FSNE shares completed materials with Federal partners and the public through the FNIC-NAL Food Stamp Nutrition Connection Web site. FSNE partners coordinate, complement and supplement other related federal, state and local programs in order to deliver consistent behavior-focused nutrition messages. The FSNE State plan submission describes the State's involvement with the State Nutrition Action Plan (SNAP) initiatives, which promotes collaboration and use of integrated nutrition education approaches across Food and Nutrition Service (FNS) programs. The State outlines any tasks in the SNAP process that FSNE will lead or conduct during the fiscal year. The result of the SNAP process is a comprehensive state plan document, including shared performance goals which apply across programs. The FSNE State Plan outlines any tasks in the SNAP process that FSNE will lead or conduct during the fiscal year. The State is required to submit with its plan a copy of a written signed agreement that clearly outlines the responsibilities of all partner agencies.(Guidance page 19, #4 bullet 2) A review of FY04 FSNE activities found extensive collaboration and coordination in FSNE. Approximately 73% of FSNE implementing agencies either had a formal partnership(s) or reported that FSNE was well-coordinated with at least one other USDA nutrition education initiative. Almost 40% of implementing agencies reported similar coordination with other (non-USDA) nutrition education efforts, including, State government, local and nonprofit and/or private organizations. Local FSNE providers also reported coordinating with others; local partners typically provided access to deliver nutrition education at their sites to low-income persons.

Evidence: Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008. Abt Associates, Food Stamp Nutrition Education Systems Review Final Report. USDA, September 2006. State Nutrition Action Plans FNIC-NAL Food Stamp Nutrition Connection Web site: www.nal.usda.gov/foodstamp.gov http://www.gao.gov/new.items/d04528.pdf

Does the program use strong financial management practices?

Explanation: Under Section 16 of the Food Stamp Act of 1977, as amended, a State food stamp agency is reimbursed 50 percent for allowable costs that are reasonable and necessary to operate nutrition education activities, thereby assuring that non-Federal sources of funds are equally invested in payment for FSNE. Nutrition education projects must be established in the fiscal year the funds are appropriated but disbursements against these obligations may occur after the fiscal year closes. Federal reimbursement is subject to the availability of funds, and is only permitted for allowable costs are defined in relevant OMB regulations. All costs charged to FNS must be valid obligations of the State, local government or sub-grantee and must be necessary and reasonable as charges under an approved FSNE plan. State Plans of Operation for each fiscal year are due by August 15 of the year prior, and are subject to FNS approval for reimbursement. When a FSNE State plan is approved, the State submits an FNS-366A Budget Projection to project funding needs. When the budget is approved, FNS notifies the State in writing of the approved amount of Federal funding. Certain financial management practices developed for FSNE have become the model for other FNS grants. Materials developed specifically for this activity are now used in other FNS grants and other federally funded grants. Specifically, alternative space computation options, time and effort reporting, and other cost areas have received significant development activities resulting in written policy guidance. FNS is currently working on other areas including multiple indirect cost rates for universities to develop a final FNS position which may become a Federal Government wide standard. Although specific audits are not available for FSNE, supportive documentation exists which evidences steps taken to assist States in addressing financial management deficiencies. These documents are based on the OMB Circulars.

Evidence: Food Stamp Nutrition Education Plan Guidance (Page 46), Federal Fiscal Year 2008OMB regulations at 2 CFR part (OMB Circular A-87), 2 CFR 220 (OMB Circular A-21), 2 CFR 215 (OMB Circular A-110) and 2 CFR 230 (OMB Circular A-122. Circulars and guidance: http://foodstamp.nal.usda.gov/nal_display/index.php?info_center=15&tax_level=2&tax_subject=250&topic_id=1506

Has the program taken meaningful steps to address its management deficiencies?

Explanation: FNS has strengthened its review and approval procedures over the past several years by bolstering its plan guidance and devoting more FNS oversight to plan review. FNS began to strengthen FSNE guidance 10 years ago when FSNE financial obligations began increasing substantially. Special emphasis has been given to the more complex issues of financial management such as time reporting, allowable costs, and documentation of in-kind contributions and complex program aspects such as targeting services to low income people. FNS also has highlighted nutrition education as an area of special interest in management evaluations over the past several years, and has asked that such state reviews be concentrated on FSNE. USDA sought to improve the effectiveness of FSNE by: Providing a clear statement of FSNE policy on measurable objectives and requirements that promote coordinated, consistent, focused efforts on improving the eating behaviors of food stamp eligibles. Key products and activities include: 1) The Guiding Principles (Appendix H Guidance)- incorporated in guidance to States- that describe the intended structure, target population, key behavioral outcomes and the role of Federal, State and local stakeholders. 2) Technical resources, assistance, and support to advance implementation of the Guiding Principles and the Guidance. These include the use of web-based Food Stamp Nutrition Connection (http://foodstamp.nal.usda.gov) to disseminate model FSNE activities, development of new materials to target key FSP populations, and creation of Statewide, collaborative cross-program plans. Providing more information on existing nutrition and promotion efforts, and an ongoing system of program monitoring to better oversee and track the quality and type of intervention activities being delivered by States Developing guidelines and providing resources for evaluating the effectiveness of FSNE activities. The issuance of an annual FSNE Plan Guidance package with cumulative policy statements resulted in a greater consistency and clarity from State nutrition education grantees. The FSNE Plan Guidance has become a model for state guidance for other FSP services, including Outreach, and Employment and Training grants. Detailed training is provided to Federal, State and local grantee staff. Since 2001, FNS has provided in-depth financial management training at 6 national conferences, as well as numerous regional venues. Although corrective actions have not been taken based on audit findings, the above cited documents indicate that strong financial practices are in place to manage fiscal aspects of FSNE.

Evidence: Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008 and prior years

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: FNS has enhanced its oversight of FSNE plans and has successively sharpened its plan guidance over the years based on past problems uncovered during plan reviews. FNS has incorporated nutrition education as a target of its state management evaluations. Problems that have been identified through this process include: inappropriate target audiences, inappropriate sources for State funding matches, duplication of effort in the development of materials or research, lack of coordination with nutrition education in other FNS programs, unreasonable costs, and disproportionate supervisory and administrative infrastructures. These issues have been addressed program-wide through changes in guidance; moreover, the specific issues identified in management evaluations have been resolved through increased FNS oversight of FSNE operations in States. FNS also annually receives final reports from each State outlining achievements, accomplishments and findings over the past year. FNS requires that each of its seven regional offices conduct two annual management evaluation (ME) reviews on select State FSNE projects. These reviews are conducted using a set of criteria (Guidance Page 4) which then allows FNS to evaluate both the fiscal and program integrity of the selected projects. Data gathered through ME reviews provides a basis for assessing the administration of FNS program operations and planning future management objectives. The Food Stamp Nutrition Connection is a web-based library for States to place curricula, materials and other FSNE developed resources. States may use the site to access other States' materials and activities to capitalize on existing resources. In addition, this site provides detailed information on the FSNE activities being implemented.

Evidence: Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008; FNS Management evaluation reports; http://www.nal.usda.gov/foodstamp/pdf/FY2007%20Approved%20Federal%20Funds-2%20pages.pdf The following chart is a sample of upcoming 2007 ME reviews scheduled for particular regions: Sample Schedule for ME Reviews: Southwest Regional Office - Texas (1/22/07); Arkansas (2/12/07) Mountain Plains Regional Office - Colorado (3/5/07); Kansas (7/9/07) Mid-Atlantic Regional Office - New Jerssey (5/14/07); District of Columbia (5/21/07) Western Regional Office - California Nutrition Network* (April - July 07); Arizona (4/23/07) * Due to the size of California's Nutrition Network, several projects are being reviewed over several months

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: While the program posts the amount of federal funding approved for each FSNE State plan on the Food Stamp Nutrition Connection, performance data is not provided. Information on FSNE funding is available publicly back to FY1992. FSNE State plans and final reports are available for review upon request. The EARS system currently being finalized will facilitate greater standardization and transparency of annual performance data on program operations and performance. In addition, the Food Stamp Nutrition Education Systems Review final report provides detailed FY2004 data on FSNE organizational structure; planning; service delivery; management and evaluation; as well as budgets and outlays. The report is available at http://www.fns.usda.gov/oane/MENU/Published/NutritionEducation/Files/FSNESystemsReview.pdf

Evidence: FNSE state plans and final reports; Food Stamp Nutrition Connection (http://www.nal.usda.gov/foodstamp/pdf/FY2007%20Approved%20Federal%20Funds-2%20pages.pdf ). EARS OMB Information Collection Approval Submission; Bell, L., Tao, F., et al. (2006) Food Stamp Nutrition Education Systems Review: Final Report. U.S. Department of Agriculture, Food and Nutrition Service

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Neither baselines nor targets have been set yet.

Evidence: Neither baselines nor targets have been set yet.

Does the program (including program partners) achieve its annual performance goals?

Explanation: While trend data on annual performance measures are not available, there is some evidence that the program has had success in achieving its annual performance goals. FSNE providers demonstrated success at targeting school-aged children and women for services in public schools. In FY 2004, an estimated 67 percent of persons receiving food stamp nutrition education were school-aged children (42 percent) or adult women (25 percent). The most frequently-used settings for direct nutrition education by FSNE implementing agencies are public schools (81%) and other youth education sites (75%). In addition, FNS is currently on schedule for EARS implementation, with the information collection approval currently pending OMB approval.

Evidence: Annual State FNSE Plans; Bell, L., Tao, F., et al. (2006) Food Stamp Nutrition Education Systems Review: Final Report. U.S. Department of Agriculture, Food and Nutrition Service.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program does not use cost-effectiveness data to drive its programmatic or funding decisions. While the agency recently developed guidance on improving FSNE State program operations, there is no data indicating State program effectiveness.

Evidence:

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: FSNE activities are explicitly designed to be integrated with State and local nutrition education efforts in order to meet the unique needs of local target groups. . Because audience characteristics, behavioral targets, and nutrition strategies vary across States and local initiatives, a centralized FSNE impact assessment would not be meaningful. While the effects of FSNE cannot be assessed separately from these State and local efforts, the coordination and integration of Federal, State and local efforts to address more general principles result in behavior-based nutrition education through multiple reinforcing channels. Given the complexity of the changes FSNE seeks, this de-centralized and tailored approach is reflection of its strength. Moreover, available data does indicate that FSNE compares favorably with EFNEP in terms of reach and targeting. EFNEP provides nutrition education to low-income populations through the county extension service. Both programs are available in virtually all States. In 2004, FSNE services were provided in most (82%) counties, while EFNEP operated in approximately one-quarter. At that time, approximately three-fourths of FSNE recipients were FSP participants or eligibles; EFNEP served a similar proportion of individuals who received assistance from one or more nutrition assistance programs.

Evidence: Food Stamp Nutrition Education Plan Guidance, Federal Fiscal Year 2008. Abt Associates, Food Stamp Nutrition Education Systems Review Final Report. USDA, September 2006. Expanded Food and Nutrition Education Program, 2004 National Impact Data, http://www.csrees.usda.gov/nea/food/efnep/pdf/2004_impact.pdf

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: FSNE operates in a diverse and decentralized manner in order to best meet the specific nutrition education needs of local target populations among food stamp recipients and eligibles. While it is difficult to design a single evaluation that could meaningfully measure the impacts of all FSNE interventions on a small set of behavioral outcomes, local evaluations of specific interventions are common. While no high quality, independent outcome evaluations have been conducted at a national level, such evaluations have been conducted on interventions which reflect large proportions of FSNE activities. The State of California (which accounts for approximately 40% of FSNE expenditures) found that California's FSNE interventions are positively associated with increased fruit and vegetable consumption reported by low-income adults, and by children living in food stamp households. This evaluation is based on data from the California Children's Healthy Eating and Exercise Practices Survey, a statewide surveillance project of the California Department of Health Services, which is conducted biennially. According to CSREES, in FY 2002, 40 States self-reported positive outcomes associated with their FSNE interventions; positive knowledge and behavior changes were most often reported in the area of diet quality. CSREES' system for ongoing monitoring and evaluating the impacts of EFNEP was recognized in the 2006 PART review for its Grants for Nutrition and Health, and is the source of the long-term performance measure for EFNEP. The PART review also noted that "Because EFNEP is the model for much of the larger FSNE program, it is indicative of the success of that program as well". Cooperative Extension Service FSNE providers, who comprised 55% of FSNE providers in FY2004, use EFNEP as a model for FSNE. CSREES reports that in FY2005, 89% of EFNEP participants reported improvements in one or more nutrition practices.

Evidence: Anliker, J., Bell, L., et al. (2000) Food Stamp Nutrition Education Study: Final Report. U.S. Department of Agriculture, Food and Nutrition Service; Bell, L., Tao, F., et al. (2006) Food Stamp Nutrition Education Systems Review: Final Report. U.S. Department of Agriculture, Food and Nutrition Service; Little, D. and Newman, M. (2003) Food Stamp Nutrition Education within the Cooperative Extension/Land-Grant University System: National Report FY 2002. U.S. Department of Agriculture, Cooperative State Research, Education, and Extension Service. http://www.csrees.usda.gov/nea/food/fsne/pdfs/fsne_national_report_2002.pdf Presentation by California Department of Health Services, Association of State Nutrition Network Administrators, Winter 2007 meeting. CalCHEEPS description: http://www.dhs.ca.gov/ps/cdic/cpns/research/download/calcheeps_2003/calcheeps_Background_Doc.pdf; CSREES 2006 PART review.