Detailed Information on the
Nutrition Assistance for Puerto Rico Assessment

Working with Puerto Rico to develop a plan for regular program review and assessment to support program improvements.

Conduct a study comparing the feasibility and impact of providing nutrition assistance in Puerto Rico through the standard Food Stamp Program structure with the current block grant program.

Working with Puerto Rico to confirm the validity of the program's performance measures, establish long-term and annual performance targets and a plan for annual progress reports, by August 1, 2006.

Working with Puerto Rico to develop a corrective action plan for any outstanding audit findings.

Measure: Customer Service Standard

Explanation:The application processing measure is defined as the proportion of all applications processed within prescribed timeframes (within 30 days for regular benefits or within 3 days for emergency benefits) to the number of all applications processed. Data on regular benefit applications is derived from the Commonwealth's automated management information system. Timeliness of emergency applications is assessed using audits of a random sample of applications. Targets are set at 95% (the standard to which State agencies in the Food Stamp Program are held). Puerto Rico is still in the process of developing its reporting systems to ensure that its performance measures have integrity and validity). USDA continues to work with the Commonwealth to confirm the validity of the NAP measures, establish long-term and annual performance targets and a plan for annual progress reports.

Measure: Payment accuracy rate

Explanation:The Commonwealth has targeted a rate of payment accuracy -- the sum of benefit over-issuances and under-issuances divided by total benefits - of 95.5% or higher (a level above the record high national average in the United States). It is measured through the Measure of Efforts and Results System (MERS), a periodic review of the circumstances of a sample of participating cases. Puerto Rico is still in the process of developing its reporting systems to ensure that its performance measures have integrity and validity). USDA continues to work with the Commonwealth to confirm the validity of the NAP measures, establish long-term and annual performance targets and a plan for annual progress reports.

Is the program purpose clear?

Explanation: The Food Stamp Act of 1977, as amended, authorizes payments to the Commonwealth of Puerto Rico for nutrition assistance programs for needy persons. The State Plan of Operation states that the purpose of the program is to provide supplemental income to the needy residents of Puerto Rico in order for them to cope with their nutritional needs.

Evidence: Section 19 of the Food Stamp Act of 1977, as amended; State Plan of Operation for the Administration of the Nutrition Assistance Program (NAP) for the Commonwealth of Puerto Rico, Fiscal Year 2005.

Does the program address a specific and existing problem, interest, or need?

Explanation: In fiscal year 2004, the program provided over $1.3 billion in nutrition assistance benefits to 1.01 million low-income people in Puerto Rico. According to the State Plan of Operation, about 59 percent of the island's population has income below the federal poverty guidelines ($18,850 for a household of four), unemployment is high (11 percent in April 2004), and work force participation is low (47 percent). The extent of hunger and malnutrition in Puerto Rico is not known because the major surveys used to measure these conditions in the United States are not conducted in the Commonwealth. Poor eating habits, coupled with sedentary lifestyle, have provoked high rates of overweight and obesity in Puerto Rico. The Food and Nutrition Commission in Puerto Rico reports that over 60 percent of adults in Puerto Rico are overweight and 22 percent are obese. In 2005, an expert panel convened by USDA reported finding that poverty, as measured by household income, is associated with obesity in some population groups.

Evidence: State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico Fiscal Year 2005; Puerto Rico Monthly Participation and Benefit Reports for Fiscal Year 2004; Linz, P., M. Lee, and L. Bell (2005). Obesity, Poverty, and Participation in Nutrition Assistance Programs. Prepared by ALTA Systems, Inc. for the Food and Nutrition Service, USDA (available on-line at www.fns.usda.gov/oane).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Like the Food Stamp Program which it replaced, the Nutrition Assistance Program (NAP) is available to nearly anyone with little income and few resources. Other Federal nutrition assistance programs operating in Puerto Rico, such as WIC, commodity assistance, and summer food service, serve specific populations in specific settings. Twenty-five percent of the NAP benefit is provided in cash which is potentially duplicative for those who receive TANF and other forms of cash assistance. However, the cash portion of the NAP benefit is necessary since it enables households without access to stores with point-of-sales devices to obtain food.

Evidence: State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico Fiscal Year 2005.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The elimination of food coupons and implementation of electronic benefits has significantly addressed a major problem in the previous Food Stamp Program. Food coupons were subject to storage and accountability problems, theft, counterfeiting and trafficking. Twenty-five percent of the NAP benefit is provided as cash, which is less effective than nutrition benefits for increasing household expenditures on food. However, the cash portion of the NAP benefit enables households without access to stores with point-of-sales devices to obtain food.

Evidence: State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico Fiscal Year 2005; Section 19 of the Food Stamp Act of 1977, as amended; USDA Food and Nutrition Service. 1983. Evaluation of Puerto Rico Nutrition Assistance Program; Beebout, H. et al. 1985. Evaluation of the Nutrition Assistance Program in Puerto Rico. Washington, DC. Mathematica Policy Research, Inc.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Only very low income households are eligible for NAP. The program serves households with resources below $2,000 ($3,000 for households with elderly or disabled) and gross annual income less than $8,004 for a family of four (income limits vary by household size.) For households with an elderly or disabled person, the program serves those with annual net income of less than $6,156. NAP reports low erroneous payments. The error rate for May through September 2003, was 3.6 percent and for October 2003 through March 2004 2.9 percent.

Evidence: State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico Fiscal Year 2005.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: NAP has three long-term performance measures: the payment accuracy rate, the participation rate among low-income elderly, and a customer service standard. NAP's purposes are aligned with two crosscutting USDA goals -- reducing the prevalence of hunger among low-income households and improving the Healthy Eating Index (a measure of dietary quality) for low-income individuals -- although the data sources for these measures do not include residents of the Commonwealth. NAP does not have outcome measures because of the complexity of measuring program impacts on participants. Since households at greater risk of hunger are more likely to seek assistance, assessment of program effectiveness requires complex analytical methods to account for differences between participants and non-participants. Even if these methods were available, the major national surveys used to measure this information are not conducted in Puerto Rico. Replicating these studies in Puerto Rico would require great time and financial resources.

Evidence: Puerto Rico Departments of Family, Education and Health, Strategic Plan for 2003-2007 (Spanish); USDA Strategic Plan 2002-2007.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: By 2007, Puerto Rico seeks to achieve a payment accuracy rate of 95 percent (no baseline identified), serve 85 percent of elderly with income below the poverty line (compared to 70 percent in 1999), and process all recipient complaints within 60 days (compared to 50 percent in 2001). While the latter two targets represent ambitious goals, the payment accuracy target is not ambitious since it does not promote continued improvement. The Commonwealth reports being below this level for many years.

Evidence: Puerto Rico Departments of Family, Education and Health, Strategic Plan for 2003-2007 (Spanish). Puerto Rico will update its strategic plan prior to the end of this fiscal year and again set outyear targets.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program's annual performance measures are the same as the long-term measures: payment accuracy, elderly participation, and customer service.

Evidence: Puerto Rico Departments of Family, Education and Health, Strategic Plan for 2003-2007 (Spanish).

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The Commonwealth has not set annual targets separate from the long-term goals.

Evidence: Puerto Rico Measure of Efforts and Results System (MERS).

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The Puerto Rico Strategic Plan was written jointly by the FNS Caribbean Area Office and the four Commonwealth Government agencies that administer FNS programs. The plan is a local initiative that establishes specific goals and objectives for each program, including NAP, and fosters collaboration among the administering agencies.

Evidence: Puerto Rico Departments of Family, Education and Health, Strategic Plan for 2003-2007 (Spanish).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Independent evaluations conducted shortly before and after implementation of the block grant in 1982 indicated that NAP resulted in fewer program participants, reduced Federal cost, and slight reductions in food expenditures compared to the Food Stamp Program that it replaced. Since that time, the program has only been reviewed occasionally. USDA is unaware of any studies that Puerto Rico may have undertaken in recent years.

Evidence: Beebout, H., et al. 1985. Evaluation of the Nutrition Assistance Program in Puerto Rico, Volume 1: Environment, Participation, Administrative Costs, and Program Integrity. Washington, DC; Mathematica Policy Research, Inc; Beebout, H., et al. 1985. Evaluation of the Nutrition Assistance Program in Puerto Rico, Volume 2: Effects on Food Expenditures and Diet Quality. Washington, DC; Mathematica Policy Research, Inc; Fox, M. K. et al. Editors. 2004. Effects of Food Assistance and Nutrition Programs on Nutrition and Health, Volume 3, pp. 286-292; USDA Food and Nutrition Service. 1983. Evaluation of the Puerto Rico Nutrition Assistance Program; USDA, Food and Nutrition Service; U.S. General Accounting Office. 1992. Food Assistance: Nutritional Conditions and Program Alternatives in Puerto Rico. Report to Congressional Committees No. GAO/RCED-92-114.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: FNS' budget request displays resources for NAP and other programs in alignment with long-term USDA strategic goals. However, the budget does not explictly tie the NAP budget request to the accomplishment of or improvement in the program's annual or long-term performance goals, nor report all direct and indirect costs needed to attain performance results.

Evidence: 2006 President's Budet. Resources requested for NAP in the budget are estimated by projected increases in the Thrifty Food Plan (TFP) and ultimately set by law by the actual changes in the TFP.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The major step the program has taken in the area of strategic planning is to develop the Puerto Rico Departments of Family, Education and Health Strategic Plan for 2003-2007. NAP is currently operating under the plan.

Evidence: Puerto Rico Departments of Family, Education and Health, Strategic Plan for 2003-2007 (Spanish).

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The block grant was established to provide broad flexibility to the Commonweath while reducing Federal expenditures. Consistent with this intention, USDA has played a limited role in addressing issues of program management. USDA collects data on participants and program costs from Puerto Rico on a monthly basis. However, it does not routinely collect data relating to the NAP program goals or play a role in managing the program to help achieve these goals. Puerto Rico operates a Quality Control-like system through which it regularly reviews a sample of cases to evaluate the quality and punctuality of services. The program is also regularly audited by the Administration for Socio-Economic Development of the Department of the Family to ensure compliance with performance standards established in annual work plans. However, Puerto Rico has not set a baseline or targets that promote continued improvement in payment accuracy. NAP also appears not to regularly report data relating to the elderly participation rate or its customer service goal.

Evidence: Puerto Rico Monthly Participation and Benefit Reports. State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico, Fiscal Year 2005.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Commonwealth is required to conduct a biennial audit of NAP expenditures and report audit findings to USDA. Puerto Rico and FNS have agreed to an annual single audit in place of the biennial audit in order to detect and correct program problems more quickly, thereby conserving program funds. The Food Stamp Act authorizes FNS to suspend payment of block grant funds if it determines that annual plans of operation are not submitted as required by law. These plans delineate how the program will operate in the fiscal year for which it is submitted. FNS approval or disapproval of the plan is influenced by past program performance and Commonwealth efforts to correct deficiencies. FNS may seek injunctive relief to enforce compliance with the plan submittal requirement. However, Federal managers do not have clearly defined or quantifiable performance standards for NAP.

Evidence: Sections 19(b)(2)(A) and (C) of the Food Stamp Act of 1977, as amended; A-133 audits and NAP State Plan of Operation.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Puerto Rico obligates NAP funds in a timely manner. FNS reviews and approves the Commonwealth's annual budget plan and monitors program operations to ensure funds are spent for the intended purpose. These monitoring activities include reviewing financial reports of obligations and expenditures, and on-site management reviews of selected program operations.

Evidence: SF-133, Report on Budget Execution and Budget; SF-269, Resources and the Financial Status Report.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: NAP has a Quality Control (QC)-like component, Measure of Efforts and Results System, which evaluates the quality and promptitude of services. The system works with a sample of nearly 1,900 NAP and TANF cases and reviews many eligibility elements. The program measures and tracks payment accuracy against its long-term performance target.

Evidence: Sections 19(b)(2)(A) and (C) of the Food Stamp Act of 1977, as amended. State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico Fiscal Year 2005.

Does the program collaborate and coordinate effectively with related programs?

Explanation: No evidence has been provided of collaboration with related programs leading to meaningful actions in management and resource allocation.

Evidence:  

Does the program use strong financial management practices?

Explanation: Puerto Rico has procedures in place to minimize erroneous payments and help to reduce fraud. The State Plan describes procedures for certification, claims collection and intentional violation penalties. Financial management systems meet the requirements of OMB Circular A-87 and of the Treasury Department. The EBT system for benefits meets the requirements of federal regulations in their reconciliation procedures. Expenditures reported on the Financial Status Report (SF-269) are submitted in a timely and accurate manner. However, the 2002 Single State Audit identified several non-compliances with financial requirements some of which may represent significant deficiencies.

Evidence: FNS received an unqualified opinion in its financial statements audit in FYs 1998 through 2000. The Agency has had no reportable conditions in the Department's consolidated financial statements audit in FYs 2001-2004; State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico Fiscal Year 2005; Single Audit Report of the Commonwealth of Puerto Rico, Department of Family, 2002.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Since the block grant was established, USDA has not had an active role in program monitoring. However, USDA has worked with NAP to effect program improvements. For example, until September 2001, all NAP benefits were provided as cash. USDA and Puerto Rico worked to require that 75 percent of NAP benefits be used at authorized food retailers and allow 25 percent to be transacted at ATM machines and other locations with point-of-sale (POS) devices. Later, USDA and Puerto Rico further tightened controls by only allowing the cash portion to be accessed through POS devices at authorized food retailers. This change in program structure strengthened the program's link to food assistance. The Commonwealth has two systems for identifying and correcting NAP management deficiencies. The first is the program's MERS system, by which the agency tracks program performance by reviewing a sample of case files. Findings are shared with local offices, which carry out corrective action, if necessary. Also, the Administration for Socio-Economic Development (ASED) of the Department of the Family regularly audits local offices to monitor their compliance with standards established in annual work plans. The program auditors review controls and procedures for the prompt delivery of services, read sample cases to verify compliance with norms, prepare and analyze local office profiles to measure performance, and provide technical assistance. They prepare findings and recommendations, and require corrective action plans where needed.

Evidence: State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico Fiscal Year 2005; Puerto Rico is working with its partners to correct problems with the Special Wage Incentive Program (SWIP), the employment component of NAP.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The Commonwealth performs an annual audit of Nutrition Assistance Program operations and submits the results to FNS. The USDA Office of Inspector General (OIG) also audits the program on occasion. FNS is in regular contact with the Puerto Rico Department of the Family concerning NAP. In addition, Puerto Rico submits an annual plan of operations, including a detailed budget, which is reviewed and approved by USDA staff. During the year, Puerto Rico submits quarterly financial reports detailing obligations and actual expenditures, and after the end of the year USDA staff conduct a financial closeout for NAP which reconciles the budget plan with reported and actual Program expenditures. Finally, USDA conducts financial management reviews in Puerto Rico every three years to obtain reasonable assurance that the financial information reported by State agencies is correct and complete; that it represents proper expenditures of Federal funds; and that States have complied with applicable financial requirements. USDA also conducts management evaluations on a similar cycle.

Evidence: Section 19(b)(2)(A) and (C) of the Food Stamp Act of 1977, as amended. State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico Fiscal Year 2005; FNS Financial Management Review (FMR) Guide; Management Evaluation and Reviews; Food Stamp Program Regulations; The last two financial management reviews completed in Puerto Rico were in January 2005 and March 2003. The reviews covered both administrative and benefit expenditures.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The Commonwealth submits program participation and benefit data to FNS on a monthly basis. However, this information is not disseminated to the public in an accessible manner. FNS has indicated that it is willing to examine the feasibility of regularly posting NAP data to the FNS website.

Evidence: Puerto Rico Monthly Participation and Benefit Reports.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: NAP reports maintaining a high level of payment accuracy, but has not set a long-term performance goal that represents continuous improvement. In May-September 2003, NAP achieved a payment accuracy rate of 96.4 percent. Information is not available on the program's progress in meeting its customer service or elderly participation goals. However, in 2004, the program began disregarding half of the amount of Social Security payments received by a household. This program change was estimated to add 42,000 elderly and disabled individuals to the program and should help achieve the long-term goal of increasing the elderly participation rate to 85 percent of eligibles.

Evidence: Measure of Efforts and Results System (MERS); State Plan of Operation for the Administrtion of the Nutrition Assistance Program for the Commonwealth of Puerto Rico.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The Commonwealth has not set annual targets against which its performance can be compared. However, performance data from May - September 2003 suggest that the program is maintaining a high payment accuracy rate.

Evidence: Measure of Efforts and Results System (MERS); State Plan of Operation for the Administrtion of the Nutrition Assistance Program for the Commonwealth of Puerto Rico.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: As of May-September 2003, NAP achieved a payment accuracy rate of 96.4 percent, above the 2007 goal of 95 percent. Information from the Commonwealth suggests that NAP maintains a high level of payment accuracy, but does not necessarily show continuous improvement each year. More information on how NAP ensures payment accuracy is needed to determine that the program clearly demonstrates very high levels of efficiency.

Evidence: State Plan of Operation for the Administration of the Nutrition Assistance Program for the Commonwealth of Puerto Rico Fiscal Year 2005.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The block grant is successful in two respects compared to the Food Stamp Program that it replaced and consistent with its legislative origins. First, it substantially reduced Federal costs (by more than $500 million a year) compared to the Food Stamp Program it replaced. Second, it substantially reduced the percentage of the Commonwealth's population that receives assistance, from 56 percent in 1981 to 26 percent in 2004. However, as a result of its flat and exceedingly low income eligibility limit, NAP is less able than the Food Stamp Program to serve many individuals living in poverty. The program serves households with income below $8,004. In 2005, this level is equal to 42 percent of the poverty level for a four-person household, leaving many individuals living well below the U.S. poverty line ineligible for benefits.

Evidence: Puerto Rico Monthly Participation and Benefit Reports GAO draft report, Nutrition Conditions and Program Alternatives in Puerto Rico, April 1992. Census estimates of population for Puerto Rico for April 1, 2004.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: NAP has not been evaluated since the 1980s. Studies from 1983 and 1985 showed that NAP achieved its goal of reducing nutrition assistance expenditures in the Commonwealth. Two years after implementation of the block grant, there were fewer households participating in NAP than in the FSP, and most households had their benefit reduced under NAP. NAP increased food expenditures compared to the absence of any nutrition assistance, but caused relatively small reductions in food expenditures and the value of food used at home by households relative to the FSP. Since that time, the program has switched to a NAP benefit, seventy-five percent of which can only be spent on food.

Evidence: Beebout, H., et al. 1985. Evaluation of the Nutrition Assistance Program in Puerto Rico, Volume 1: Environment, Participation, Administrative Costs, and Program Integrity. Washington, DC. Mathematica Policy Research, Inc; Beebout, H., et al. 1985. Evaluation of the Nutrition Assistance Program in Puerto Rico, Volume 2: Effects on Food Expenditures and Diet Quality. Washington, DC. Mathematica Policy Research, Inc; Fox, M. K. et al. Editors. 2004. Effects of Food Assistance and Nutrition Programs on Nutrition and Health, Volume 3, pp. 286-292; USDA Food and Nutrition Service. 1983. Evaluation of the Puerto Rico Nutrition Assistance Program. USDA, Food and Nutrition Service; U.S. General Accounting Office. 1992. Food Assistance: Nutritional Conditions and Program Alternatives in Puerto Rico. Report to Congressional Committees No. GAO/RCED-92-114.