Measuring Results of the Burkina Faso Access to Rural Finance Activity

December 16, 2016

In Context

The MCC compact with Burkina Faso was a five-year investment (2009-2015) of $480.5 million in 4 projects: : the Agriculture Development Project, the Rural Land Governance Project, the Roads Project, and the Burkinabé Response to Improve Girls’ Chances to Succeed (BRIGHT II) Project. The Agricultural Development Project (ADP) included 3 major activities: Water Management and Irrigation, Diversified Agriculture, and Access to Rural Finance.

The Access to Rural Finance activity consisted of 3 components: Establishment of a Rural Finance Facility, Capacity Building for Participating Financial Institutions (PFIs), and Business Development Services (BDS).

The $6.7 million Access to Rural Finance Activity is the subject of both the results described here and an independent impact evaluation (found here) released by MCC in January of 2016. This component represents 1.4% percent of the total compact. Other components of the compact are the subject of forthcoming independent evaluations.

Program Logic

The Access to Rural Finance Activity was designed to address the lack of financing options for agricultural projects in Burkina Faso.

There were several key assumptions underlying the Access to Rural Finance Activity program logic during the design of the investment:

This activity provided credit to financial institutions at a subsidized rate, assuming this would promote medium term lending to relatively risky small and medium sized rural agricultural enterprises
That the Access to Rural Finance Activity would be integrated into the larger Agriculture Development Project and be targeted at the same population as the rest of the Project in order to support the overarching goal of increasing agricultural investment and production.

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources. Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program. However, monitoring data is limited in that it cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention. The limitations of monitoring data is a key reason why MCC invests in independent impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone. Where estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

Monitoring Results

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

Indicators	Level	Baseline (2009)	Actual Achieved (07/2014)	Target	Percent Complete
Value of Agricultural and Rural Loans (in millions of US$)	Output	0	2.8	5	56%
Loan Borrowers	Output	0	96	120	80%
Firms and Farmer Groups Trained in Credit	Output	0	199	160	124%
Potential Rural Finance Facility Borrowers Capacity Reinforced in Preparing Loan Applications	Output	0	283	750	38%
Number of Borrowers Accessing Credit After Receiving Support in Developing their Loan Applications	Outcome	0	96	100	96%

Source: (July 2014 ITT, based on reporting from Access to Rural Finance Contractors)

The average completion rate of output and outcome targets is 78.8% percent; and in 1 of the 5 output or outcome indicators that included targets, the targets were met or exceeded.

Implementation of the Access to Rural Finance Activity (ARF) experienced a number of delays and difficulties in implementation. In 2013, MCA and MCC decided to lower the achievement targets to take the delays and difficulties into account (these reduced targets are those represented in the above table).

By July 2013, it became clear to MCC that two key issues impeded the ability of the Activity to achieve even these reduced targets: 1- The limited amount of time remaining prior to the end of the Compact, and 2- limited uptake in the various ARF services. In response, it was decided to suspend the Activity early rather than continue to expend resources on an Activity that was not achieving its intended goals.

Evaluation Questions

The evaluation was designed to answer questions such as:

The validity of the program logic and its assumptions (Project Design)
The degree to which planned activities were implemented (Project Implementation)
Outcome results (Outcomes)
Lessons learned

Evaluation Results

Given that the Access to Rural Finance (ARF) Activity was not able to meet its short-term output goals and was terminated early, it was not expected that significant progress towards anticipated longer-term outcomes would have been achieved. Thus, though the evaluation did attempt to respond to questions regarding outcome results, the primary focus was on examining project design and implementation, as well as on lessons learned.

Only limited quantitative data was available from participating banks for the evaluators to review. As a result, in addition to a user survey, the evaluation utilized primarily qualitative evidence gathered through interviews and focus groups for the majority of its findings.

Evaluator	A2F
Methodology	Mixed-Methods Performance Evaluation
Evaluation Period	2015
Project Design	The ARF project design failed to build on best practices in the field of rural/agricultural finance, including the completion of a value chain analysis to inform design The target market for end borrowers was too broad, leading to two main effects: 1- PFIs continued to rely on their standard selection criteria, which meant that those who received loans likely would have been those who would have received loans without the project, and 2- a lack of overlap between ARF components such that, in many cases, those who received loans were not those who had participated in the Business Development Services (a finding which was contrary to the monitoring data) The incentives to encourage PFI participation (primarily that of subsidized interest rate on capital) proved insufficient to encourage PFIs to overcome the administrative burdens and problematic delays during implementation
Project Implementation	Delayed signature of the AD10 contract to implement the ARF Activity caused significant delays in project startup The complexity of the AD10 contract, which also covered other parts of the Agriculture Development Project, was difficult for the MCA to manage, and there was a lack of sufficient staff to provide oversight

Administrative procedures were cumbersome, leading to numerous delays and discouraged interest on the part of PFIs

Outcomes

Neither outputs nor shorter-term outcomes achieved their initial targets
In many cases, those who did receive loans were likely to have been those who would have received traditional loans, without the project
Despite the Activity’s shortcomings, however, the banks who participated as PFIs did appear to benefit somewhat from the Activity, as two of the three note that they have since been trying to expand their products to better meet agricultural lending needs; expansion of agricultural lending within Burkina Faso was one of the longer-term objectives of the Activity

Lessons Learned

In addition to examining project design, project implementation, and project outcomes, the evaluation also identified key lessons to be learned from the Access to Rural Finance Activity. Among the lessons to be learned are:

A thorough needs assessment must be completed in order to ensure that the needs and capacities of sector actors are taken into account and to ensure that incentives are properly aligned across sector actors and that they are sufficient to bring about the project-required outputs
Benefits of prior experience and best practices from the available sector literature should be incorporated into project design
Consumer protection safeguards should be put in place to avoid potential abuses and/or the provision of misleading information to end users
Specific, sector experts need to be hired; both by MCC and by the MCA in order to provide sufficient oversight and direction

Next Steps

This evaluation is the Final Evaluation for the Access to Rural Finance Activity. Thus, no future evaluations will be conducted for this particular activity. However, the remaining two activities of the Agricultural Development Project- Water Management and Irrigation and Diversified Agriculture- will be evaluated via a separate evaluation. Final evaluation reports from this evaluation are anticipated in 2018.