Final Evaluation of AGRA Market Access Program's Phase I

The purpose of the evaluation is to assess the overall program performance and to provide feedback for new strategic directions or implementation designs and delivery. The evaluation will assess what has worked, what did not work so well, key lessons learned and what should be emphasized or adjusted, in the second phase. This will essentially cover the analysis of the process of implementation, the changes that have occurred as a result of the program’s intervention, opportunities and constraints that have been encountered, important lessons that have been learnt and recommendations for future design and implementation

In order to objectively achieve the purpose above, the evaluation will be guided by the following ethical rules/considerations:

• Openness – of information given, to the highest possible degree to all involved parties
• Broad participation – the key stakeholders should be involved when relevant/possible
• Reliability and independence – the evaluation should be conducted so that findings and conclusions are correct and trustworthy.

The specific evaluation objectives include:

1. Determining the relevance or appropriateness of the Program theory of change, design, strategies, management structure and delivery mechanisms and assessing whether or not they are adequate to realize the intended program objectives;
2. Critically assessing to what extent, if any, the program achieved its intended objectives; outputs, and whether these are beginning to yield into outcomes, both intended and unintended;
3. Ascertaining to what extent the program was effective and efficient in achieving its intended objectives;
4. Assessing whether the program has been cost effective in achieving the desired outcomes and the likelihood that those results will be sustained over the medium to long term;
5. Identifying to what extent the Markets Program integrates and aligns with other AGRA programs, the quality of partnerships built, if any, with relevant national and regional institutions and other stakeholders in supporting program delivery and whether these are sustainable over time;
6. Assessing the efficacy of the different models utilized by the program, for instance, building the capacity of market access actors, improving infrastructure, promoting produce quality through improved post-harvest handling practices and using the output market structures to facilitate access to finance for smallholders.
7. Identifying challenges and documenting lessons learned and best practices that will inform future strategic program decisions.
8. The evaluation will also provide recommendations on practical strategic actions that need to be taken to improve future implementation and ensure sustainability of outcomes and institutionalization of key lessons learnt. Suggestions on how to build stronger governance systems, gender integration, and how future program initiatives should feed into ongoing and planned national-level agricultural strategies to enhance synergy, complementarity and value addition will be expected.

Scope and focus of the Evaluation

The evaluation will essentially cover the following thematic areas:

1. Programmatic impacts – what outputs have been delivered, what changes (outcomes, both intended and unintended) have occurred among the beneficiaries, relevance of program interventions, return on investments and the sustainability of the achievement to-date;
2. Program delivery mechanisms – the approach adopted by the program to achieve the results highlighted above including models/best practices employed by the program; e.g. the warehouse receipt system, support for aggregation centers, linkage of smallholder farmers with structured markets, etc.
3. Program management for results- This would look at the internal program management processes to assess their adequacy, efficiency and effectiveness. These will include the program structure; staffing; grant-making processes; quality of grants funded; level of integration and alignment with other AGRA programs; and how the program has supported public and private partnerships to facilitate market access for smallholders.

The Evaluation Questions

The following major areas and issues should be addressed in the evaluation questions:

• Program performance and Impact

1. Extent to which program outputs, outcomes and objectives were achieved at all levels;
2. Positive and negative, direct and indirect, planned and unplanned results of the program;
3. Effectiveness of program and project activities;
4. The strategic importance of the program achievements and how it interfaces with the next phase of the program tenure;
5. Document challenges, lessons learned and best practices in program, sub-program and project implementation

• Efficiency in resource utilization

1. The use of resources against budgets;
2. Value - for - money analysis of program results to assess the quality of results/deliverables against the cost of achieving these results in comparable contexts;
3. Potential and options for future sustainability, scalability and replicability of achievements.

• Management issues

1. Management issues such as the overall organizational structure of the Market Access Program, staff configuration, execution modalities, and quality of grant making and management approach, arrangements for implementing grants, capacity gaps, utilization of the expertize of consultants and how they affected program results.

• Lessons learned

1. Menu of portfolio investments e.g. capacity building, infrastructure development, equipment supplies, promotion of post-harvest management technologies etc. How can these investments be sharpened and up-scaled?
2. What has been learned and what can be built upon in the next phase of the program?
3. How did the lessons learned during the Program implementation help to improve program investments and management in the future?

Methodology for Evaluation

While a detailed survey design and methodology to achieve the objectives of the end of program evaluation shall be defined by the consultant, and approved by AGRA, the following is broadly suggested as a guide. The Evaluation methodology will be a combination of qualitative and quantitative approaches for data collection and analysis. Essentially, this will entail: Desk review of key program documents and program implementation progress reports from grantees, program officers and other partners. Such documents shall include but not limited to the background program documents; grant proposals; progress reports; projects’ rapid assessment reports, and other key documents related to the program. Besides the review of relevant literature related to the assignment, the consultants shall also undertake field data collection using structured interviews with key informants, notably; farmer organizations, agro-dealer networks, project administrative and technical personnel, program partners/and donors, AGRA program staff and management; and targeted Focus Group Discussions (FGDs), as appropriate.

To enrich the qualitative data obtained through the above qualitative methods, the Consultants shall also administer questionnaires to a representative sample size of farmer households, SMEs, and other relevant stakeholders. To enhance likelihood of achieving the evaluation objectives stated above, the selection of grants shall be done from a sampling frame of Market access program-funded projects which shall be clustered into: projects that have ended; projects half-way into implementation; and projects that are in the early stages of implementation. Furthermore, the consultants shall work closely with the Market Access and the M&E staff to ensure effective coverage of the projects, countries, ecological zones and cropping systems. Each of the components of the evaluation highlighted above shall adopt its appropriate methodology that best offers higher chances to obtain sufficient data to address its research questions. It is expected that the consultants will develop and institute an in-field quality control mechanism to ensure robust and valid data is collected and analyzed for reporting.

Deliverables

The consultant (s) shall be expected to prepare and submit to the client a set of key reports in the course of undertaking this assignment. These reports shall be reviewed and accepted by the client before payment is approved. The following have been identified as key reports to be submitted:

1. An Inception Report – This shall be prepared and submitted within two weeks after the signing of the contract. The consultant will prepare this after reviewing key technical documents and after discussion with the client. The inception report shall focus on: the understanding of the Terms of Reference and scope, the relevant methodology to be adopted, the evaluation design and key questions, and, the work-plan for the assignment. The inception report shall be reviewed by both the M&E and the program’s team within 5 days after submission, and shall have to be approved before proceeding to the next phase.
2. Progress brief - While there is no formal progress report required during the assignment implementation, between inception and Draft report submission, the consultant (s) shall be expected to regularly (bi-weekly) share with the client, key emerging issues and trends to avoid surprises or misconceptions by either party.
3. Draft Report – This shall be prepared and submitted to the client towards the end of the assignment. The draft report shall require feedback in form of comments, questions and inputs from the client. In addition, the consultant (s) will be required to present the Draft Report to a wider AGRA audience for validation.
4. Final Report – This shall be no more than 40 pages (excluding annexes), and submitted to the client on, or before the expiry of the assignment contract. Any valid extension may be mutually agreed between the Consultant (s) and the Client, provided it carries no extra cost to the latter. The following will also be expected from the Consultant:

• A master copy of the final report suitable for reproduction, and four copies, in full-color and bound, as well as soft copies.
• Submission of the final report, after incorporating the comments/inputs on the presented draft report. The final report shall include actionable recommendations
• All data-sets (in SPSS/Stata) and questionnaires used during the assignment shall be a property of AGRA, and shall be the responsibility of the consultant to carefully deliver them to AGRA.

Management and Reporting

To ensure consistency, enhance utilization and avoid any surprises, the firm (consultant) shall in the course of the evaluation, be required to regularly provide an update on the evaluation progress. Reporting shall be directly to the M&E Director, and in close working relationship with the Market Access Director and Program Officers.

Timing

The evaluation is expected to be completed within a 90-day period, effective from the date a contract is signed.

In-house Resources

Access to project documents, progress and evaluation reports, program databases, contacts for grantees, financial records and other program related files, depending on the consultant’s requirements will be availed. Access to the Program Directors and AGRA staff shall be guaranteed upon request.

Documents to be provided include:

• The Program Business Plan
• Program Strategy
• Market Access M&E plan
• Program proposals to donors
• Grantee proposals and reports
• Annual investment plans
• List and contacts of Market Access Grantees
• Program progress reports
• Field trip reports
• Information available on the AGRA website
• Other publications, documentaries, success stories, studies and reports produced by the program e.g. Do all roads lead to markets, AASR

The Evaluation Team

This evaluation is expected to be conducted by a team of experts with experience in smallholder market access programs.

The team should be composed of experts with the following profiles and qualifications:

• Solid experience in evaluation of Agriculture development programs for the team leader and most of the experts. Technical knowledge of, and familiarity with, the evaluation methodology.
• Proven ability to both assess past effectiveness and provide strong strategic thinking on future direction.
• Experience working in Africa
• Full working knowledge of English, French and Portuguese as well as excellent report writing skills.
• Demonstrated ability to generate high quality, rich, readable products on time and in line with expected deliverables.
• Fully conversant with the principles and working methods of project/program cycle management
• At least one of the experts should have hands-on experience with market access related initiatives in staple value chains
• Sensitivity to local beliefs, manners, and customs and ability to act with integrity and honesty in interactions with stakeholders.

Proposal

The proposal should include:

1. A detailed elaboration of issues to be addressed/covered;
2. A description of the evaluation plan (see annex 1) including details of the proposed methodology, sampling, study design; analysis and reporting, and milestones for the evaluation and a timetable of activities.
3. Detailed budget
4. Description of the pay schedule for the review
5. Past performance summaries (at least three brief descriptions of past or current contracting mechanisms for assignments similar in size, scope and complexity to this tender) and list of references that demonstrate performance in conducting similar evaluations
6. At least one previous relevant report and list of previous reports
7. CVs conforming to the qualifications listed above for persons to manage and conduct the evaluation. Only CVs strictly relevant to the assignment shall be accepted.
8. Supporting documents includingmandatory institutional documents such as incorporation papers and most recent financial statements.

Any request for additional background information may be sent to BBamanya@agra.org

For more information on AGRA kindly visit the website: www.agra.org

How to apply:

Interested parties are requested to submit their technical and financial proposals (ensure the two are separate documents) to: procurement@agra.org by 14th November 2014