Measuring Results of the Burkina Faso Access to Rural Finance Activity
December 16, 2016
In Context
The MCC compact with Burkina Faso was a five-year investment (2009-2015) of $480.5 million in 4 projects: : the Agriculture Development Project, the Rural Land Governance Project, the Roads Project, and the Burkinabé Response to Improve Girls’ Chances to Succeed (BRIGHT II) Project. The Agricultural Development Project (ADP) included 3 major activities: Water Management and Irrigation, Diversified Agriculture, and Access to Rural Finance.
The Access to Rural Finance activity consisted of 3 components: Establishment of a Rural Finance Facility, Capacity Building for Participating Financial Institutions (PFIs), and Business Development Services (BDS).
The $6.7 million Access to Rural Finance Activity is the subject of both the results described here and an independent impact evaluation (found here) released by MCC in January of 2016. This component represents 1.4% percent of the total compact. Other components of the compact are the subject of forthcoming independent evaluations.
Program Logic
The Access to Rural Finance Activity was designed to address the lack of financing options for agricultural projects in Burkina Faso.
There were several key assumptions underlying the Access to Rural Finance Activity program logic during the design of the investment:
- This activity provided credit to financial institutions at a subsidized rate, assuming this would promote medium term lending to relatively risky small and medium sized rural agricultural enterprises
- That the Access to Rural Finance Activity would be integrated into the larger Agriculture Development Project and be targeted at the same population as the rest of the Project in order to support the overarching goal of increasing agricultural investment and production.
Measuring Results
MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources. Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program. However, monitoring data is limited in that it cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention. The limitations of monitoring data is a key reason why MCC invests in independent impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone. Where estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.
Monitoring Results
The following table summarizes performance on output and outcome indicators specific to the evaluated program.
Indicators |
Level |
Baseline (2009) |
Actual Achieved (07/2014) |
Target |
Percent Complete |
Value of Agricultural and Rural Loans (in millions of US$) |
Output |
0 |
2.8 |
5 |
56% |
Loan Borrowers |
Output |
0 |
96 |
120 |
80% |
Firms and Farmer Groups Trained in Credit |
Output |
0 |
199 |
160 |
124% |
Potential Rural Finance Facility Borrowers Capacity Reinforced in Preparing Loan Applications |
Output |
0 |
283 |
750 |
38% |
Number of Borrowers Accessing Credit After Receiving Support in Developing their Loan Applications |
Outcome |
0 |
96 |
100 |
96% |
Source: (July 2014 ITT, based on reporting from Access to Rural Finance Contractors)
The average completion rate of output and outcome targets is 78.8% percent; and in 1 of the 5 output or outcome indicators that included targets, the targets were met or exceeded.
Implementation of the Access to Rural Finance Activity (ARF) experienced a number of delays and difficulties in implementation. In 2013, MCA and MCC decided to lower the achievement targets to take the delays and difficulties into account (these reduced targets are those represented in the above table).
By July 2013, it became clear to MCC that two key issues impeded the ability of the Activity to achieve even these reduced targets: 1- The limited amount of time remaining prior to the end of the Compact, and 2- limited uptake in the various ARF services. In response, it was decided to suspend the Activity early rather than continue to expend resources on an Activity that was not achieving its intended goals.
Evaluation Questions
The evaluation was designed to answer questions such as:
- The validity of the program logic and its assumptions (Project Design)
- The degree to which planned activities were implemented (Project Implementation)
- Outcome results (Outcomes)
- Lessons learned
Evaluation Results
Given that the Access to Rural Finance (ARF) Activity was not able to meet its short-term output goals and was terminated early, it was not expected that significant progress towards anticipated longer-term outcomes would have been achieved. Thus, though the evaluation did attempt to respond to questions regarding outcome results, the primary focus was on examining project design and implementation, as well as on lessons learned.
Only limited quantitative data was available from participating banks for the evaluators to review. As a result, in addition to a user survey, the evaluation utilized primarily qualitative evidence gathered through interviews and focus groups for the majority of its findings.
Evaluator |
A2F |
Methodology |
Mixed-Methods Performance Evaluation |
Evaluation Period |
2015 |
Project Design |
|
Project Implementation |
|
|
|
Outcomes |
|
Lessons Learned
In addition to examining project design, project implementation, and project outcomes, the evaluation also identified key lessons to be learned from the Access to Rural Finance Activity. Among the lessons to be learned are:
- A thorough needs assessment must be completed in order to ensure that the needs and capacities of sector actors are taken into account and to ensure that incentives are properly aligned across sector actors and that they are sufficient to bring about the project-required outputs
- Benefits of prior experience and best practices from the available sector literature should be incorporated into project design
- Consumer protection safeguards should be put in place to avoid potential abuses and/or the provision of misleading information to end users
- Specific, sector experts need to be hired; both by MCC and by the MCA in order to provide sufficient oversight and direction
Next Steps
This evaluation is the Final Evaluation for the Access to Rural Finance Activity. Thus, no future evaluations will be conducted for this particular activity. However, the remaining two activities of the Agricultural Development Project- Water Management and Irrigation and Diversified Agriculture- will be evaluated via a separate evaluation. Final evaluation reports from this evaluation are anticipated in 2018.